{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# default_exp callback.core"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Callback\n",
    "\n",
    "> Miscellaneous callbacks for timeseriesAI."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#export \n",
    "from tsai.imports import *\n",
    "from tsai.utils import *\n",
    "from tsai.data.preprocessing import *\n",
    "from tsai.data.transforms import *\n",
    "from tsai.models.layers import *\n",
    "from fastai.callback.all import *"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#export\n",
    "import torch.multiprocessing\n",
    "torch.multiprocessing.set_sharing_strategy('file_system')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Events"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A callback can implement actions on the following events:\n",
    "* before_fit: called before doing anything, ideal for initial setup.\n",
    "* before_epoch: called at the beginning of each epoch, useful for any behavior you need to reset at each epoch.\n",
    "* before_train: called at the beginning of the training part of an epoch.\n",
    "* before_batch: called at the beginning of each batch, just after drawing said batch. It can be used to do any setup necessary for the batch (like hyper-parameter scheduling) or to change the input/target before it goes in the model (change of the input with techniques like mixup for instance).\n",
    "* after_pred: called after computing the output of the model on the batch. It can be used to change that output before it's fed to the loss.\n",
    "* after_loss: called after the loss has been computed, but before the backward pass. It can be used to add any penalty to the loss (AR or TAR in RNN training for instance).\n",
    "* before_backward: called after the loss has been computed, but only in training mode (i.e. when the backward pass will be used)\n",
    "* after_backward: called after the backward pass, but before the update of the parameters. It can be used to do any change to the gradients before said update (gradient clipping for instance).\n",
    "* after_step: called after the step and before the gradients are zeroed.\n",
    "* after_batch: called at the end of a batch, for any clean-up before the next one.\n",
    "* after_train: called at the end of the training phase of an epoch.\n",
    "* before_validate: called at the beginning of the validation phase of an epoch, useful for any setup needed specifically for validation.\n",
    "* after_validate: called at the end of the validation part of an epoch.\n",
    "* after_epoch: called at the end of an epoch, for any clean-up before the next one.\n",
    "* after_fit: called at the end of training, for final clean-up."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Learner attributes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When writing a callback, the following attributes of Learner are available:\n",
    "\n",
    "* **model**: the model used for training/validation\n",
    "* **data**: the underlying DataLoaders\n",
    "* **loss_func**: the loss function used\n",
    "* **opt**: the optimizer used to udpate the model parameters\n",
    "* **opt_func**: the function used to create the optimizer\n",
    "* **cbs**: the list containing all Callbacks\n",
    "* **dl**: current DataLoader used for iteration\n",
    "* **x/xb**: last input drawn from self.dl (potentially modified by callbacks). xb is always a tuple (potentially with one element) and x is detuplified. You can only assign to xb.\n",
    "* **y/yb**: last target drawn from self.dl (potentially modified by callbacks). yb is always a tuple (potentially with one element) and y is detuplified. You can only assign to yb.\n",
    "* **pred**: last predictions from self.model (potentially modified by callbacks)\n",
    "* **loss**: last computed loss (potentially modified by callbacks)\n",
    "* **n_epoch**: the number of epochs in this training\n",
    "* **n_iter**: the number of iterations in the current self.dl\n",
    "* **epoch**: the current epoch index (from 0 to n_epoch-1)\n",
    "* **iter**: the current iteration index in self.dl (from 0 to n_iter-1)\n",
    "\n",
    "The following attributes are added by TrainEvalCallback and should be available unless you went out of your way to remove that callback:\n",
    "* **train_iter**: the number of training iterations done since the beginning of this training\n",
    "* **pct_train**: from 0. to 1., the percentage of training iterations completed\n",
    "* **training**: flag to indicate if we're in training mode or not\n",
    "\n",
    "The following attribute is added by Recorder and should be available unless you went out of your way to remove that callback:\n",
    "* **smooth_loss**: an exponentially-averaged version of the training loss"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Gambler's loss: noisy labels"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#export\n",
    "class GamblersCallback(Callback):\n",
    "    \"A callback to use metrics with gambler's loss\"\n",
    "    def after_loss(self): self.learn.pred = self.learn.pred[..., :-1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: left;\">\n",
       "      <th>epoch</th>\n",
       "      <th>train_loss</th>\n",
       "      <th>valid_loss</th>\n",
       "      <th>accuracy</th>\n",
       "      <th>time</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>1.395603</td>\n",
       "      <td>1.523850</td>\n",
       "      <td>0.266667</td>\n",
       "      <td>00:05</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from tsai.data.all import *\n",
    "from tsai.models.InceptionTime import *\n",
    "from tsai.models.layers import *\n",
    "dsid = 'NATOPS'\n",
    "X, y, splits = get_UCR_data(dsid, return_split=False)\n",
    "tfms = [None, Categorize()]\n",
    "dsets = TSDatasets(X, y, tfms=tfms, splits=splits)\n",
    "dls = TSDataLoaders.from_dsets(dsets.train, dsets.valid, bs=[64, 128])\n",
    "loss_func = gambler_loss()\n",
    "learn = Learner(dls, InceptionTime(dls.vars, dls.c + 1), loss_func=loss_func, cbs=GamblersCallback, metrics=[accuracy])\n",
    "learn.fit_one_cycle(1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Transform scheduler"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# export\n",
    "class TransformScheduler(Callback):\n",
    "    \"A callback to schedule batch transforms during training based on a function (sched_lin, sched_exp, sched_cos (default), etc)\"\n",
    "    def __init__(self, schedule_func:callable, show_plot:bool=False): \n",
    "        self.schedule_func,self.show_plot = schedule_func,show_plot\n",
    "        self.mult = []\n",
    "\n",
    "    def before_fit(self):\n",
    "        for pct in np.linspace(0, 1, len(self.dls.train) * self.n_epoch): self.mult.append(self.schedule_func(pct))\n",
    "        # get initial magnitude values and update initial value\n",
    "        self.mag = []\n",
    "        self.mag_tfms = []\n",
    "        for t in self.dls.after_batch: \n",
    "            if hasattr(t, 'magnitude'):\n",
    "                self.mag.append(t.magnitude)\n",
    "                t.magnitude *= self.mult[0]\n",
    "                self.mag_tfms.append(t)\n",
    "\n",
    "    def after_batch(self):\n",
    "        if self.training and len(self.mag_tfms)>0 and self.train_iter < len(self.mult):\n",
    "            # set values for next batch\n",
    "            for t,m in zip(self.mag_tfms, self.mag): \n",
    "                t.magnitude = m * self.mult[self.train_iter]\n",
    "                \n",
    "    def after_fit(self):\n",
    "        if self.show_plot and self.mult != [] and len(self.mag_tfms)>0: \n",
    "            print()\n",
    "            plt.plot(self.mult)\n",
    "            plt.title('Scheduled tfms')\n",
    "            plt.show()\n",
    "            print()\n",
    "            self.show_plot = False\n",
    "        # set values to initial values\n",
    "        for t,m in zip(self.mag_tfms, self.mag): t.magnitude = m\n",
    "    \n",
    "    def __repr__(self):\n",
    "        return f'{self.__class__.__name__}({self.schedule_func})'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "TransformScheduler(<fastai.callback.schedule._Annealer object at 0x7ffc9d32bdd8>)"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "TransformScheduler(SchedCos(1, 0))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAbvklEQVR4nO3deXTU9b3/8ed7JhsECEKCLEkIYZFFETEggoi2VoFa6OICFltbW7XW9t6rpz167fazv/56W39tb1ttb/FoFyviUmu5lRZbpUdFQAIoq2hkTVgStgBCAkne949M70lpIAOZme8sr8c5OWdmvp/MvL4mefnlu3y+5u6IiEjqCwUdQEREYkOFLiKSJlToIiJpQoUuIpImVOgiImkiK6gPLiws9LKysqA+XkQkJa1cuXKvuxe1tyywQi8rK6OysjKojxcRSUlmtu1Uy7TLRUQkTajQRUTShApdRCRNqNBFRNKECl1EJE10WOhm9piZ1ZrZulMsNzP7iZlVmdkaMxsb+5giItKRaLbQfwVMPc3yacDQyNdtwM87H0tERM5Uh+ehu/srZlZ2miEzgd946zy8y8ysp5n1c/ddsQrZ1oqt+3n1nbp4vLWcxMwIh4yssDGwVz5D+nSjrLAruVnhoKOJSDticWHRAGBHm+fVkdf+qdDN7DZat+IpLS09qw9bte0AP11cdVbfK9E71TT5WSFj8tBCPja2mKtHnktetspdJFkk9EpRd58LzAWoqKg4qztr3D5lMLdPGRzTXNI+d8cdGpqa2bL3fapqj7Cupp4/rtnFl59cTbfcLD4zqYw7pgwmPzewi45FJCIWf4U1QEmb58WR1yTFmRlm0DUni1H9CxjVv4CZYwZw37QRLNu8jyeWb+enL1cxf8UO7vnQMK6vKCEcsqBji2SsWJy2uAD4VORslwlAfbz2n0tyCIWMiUMKefiTY3nuzomUnNOFe59by02PLKP2UEPQ8UQyVjSnLT4JLAXOM7NqM7vVzO4wszsiQxYCm4Eq4BHgzrillaQztvQcfveFiXz/utGsqa5n+k9e5fWqvUHHEslIFtRNoisqKlyzLaaXd/Yc5s4nVvFe3RHunz6Cz00uDzqSSNoxs5XuXtHeMl0pKjEz7Nzu/OGLk5g6qi//94WN/PDFTQS1wSCSiVToElP5uVk8dNNYbqgo5icvV/HtP25UqYskiM41k5gLh4z/+PhouuZk8diSLTS1tPB/ZozCTGfAiMSTCl3iIhQyvvmRkWSHjUde3ULfgjzuvGJI0LFE0poKXeLGzLhv2gj2HGrk+3/eRL+CPD52UXHQsUTSlgpd4ioUMh68fjR1hxv56rNr6NM9j0lDCoOOJZKWdFBU4i43K8x/3Xwxgwrz+eK8VdQcPBZ0JJG0pEKXhCjoks0vbq6gudm584lVHG9qCTqSSNpRoUvCDCrM58HrR/PWjoP8v4Ubg44jknZU6JJQU8/vx62XDeJXr2/lj2t2Bh1HJK2o0CXh7p02nLGlPbnvubXsqtf+dJFYUaFLwmWHQ/zoxjE0NTtffXaNriQViREVugRiYO98/n36cF59dy9PvrGj428QkQ6p0CUwn7xkIJOG9OY7L2xgx/6jQccRSXkqdAlMKGR87xOjMTPtehGJARW6BKr4nK7cN304Szfv4/k3dedCkc5QoUvgZo8r5cKSnnznhY3UHzsRdByRlKVCl8CFQsZ3Pno++98/zg9e3BR0HJGUpUKXpHD+gAI+dWkZjy/bxtrq+qDjiKQkFbokjbuvHkZht1y+9vxaWlp0gFTkTKnQJWn0yMvm36cP563qeh0gFTkLKnRJKjMvHMDo4gIeXLSJY8ebg44jklJU6JJUQiHj/ukj2FXfwKOvbQ46jkhKUaFL0rmkvDfXjDqXn//tPWoPNwQdRyRlqNAlKd07bQSNTS386C/vBh1FJGWo0CUpDSrM5+ZLB/LUiu1U1R4JOo5ISlChS9K668ohdMkO86O/vhN0FJGUoEKXpNW7Wy6fvWwQL6zZxfqduthIpCMqdElqn5tcTo+8LH74orbSRTqiQpekVtAlm9unDOalt2tZtf1A0HFEkpoKXZLeLRPLKOyWo4m7RDoQVaGb2VQz22RmVWZ2bzvLS81ssZmtNrM1ZjY99lElU+XnZvGFK4awpGofyzfvCzqOSNLqsNDNLAw8DEwDRgKzzWzkScO+Bjzt7hcBs4CfxTqoZLabxpdS2C2HhxZXBR1FJGlFs4U+Hqhy983ufhyYD8w8aYwDPSKPC4CdsYsoAl1ywnx+cjmvvruX1dqXLtKuaAp9AND2tuzVkdfa+hYwx8yqgYXAl9p7IzO7zcwqzayyrq7uLOJKJvvkhIH07JrNQy9rK12kPbE6KDob+JW7FwPTgcfN7J/e293nunuFu1cUFRXF6KMlU3TLzeLWSYN46e1a1tXovHSRk0VT6DVASZvnxZHX2roVeBrA3ZcCeUBhLAKKtPXpSWV0z8vSVrpIO6Ip9BXAUDMbZGY5tB70XHDSmO3ABwHMbAStha59KhJzPfKyuWViGX9ev5t39xwOOo5IUumw0N29CbgLWARspPVslvVm9oCZzYgMuwf4vJm9BTwJ3OLuuoeYxMVnJg0iLzvE3Fc0X7pIW1nRDHL3hbQe7Gz72jfaPN4ATIptNJH29crP4caKEua9sZ27rx5Gv4IuQUcSSQq6UlRS0ucml9Pi8MslW4OOIpI0VOiSkkp6deXDF/Rj3vLt1B87EXQckaSgQpeUddvl5RxpbGLe8u1BRxFJCip0SVnnDyhg8tBCHluyhcam5qDjiAROhS4p7fbLB1N3uJE/rNZsEyIqdElpk4b0Znjf7jy2ZAs6U1YynQpdUpqZ8dnLBvH27sO8/p6m1pXMpkKXlDfjwv4Udsvh0de2BB1FJFAqdEl5edlh5kwYyMtv1/Je3ZGg44gERoUuaWHOhIHkZIX45RJtpUvmUqFLWijslstHx/TndytrOHj0eNBxRAKhQpe08dnLBnHsRDPzV+zoeLBIGlKhS9oY3rcHE8p78fjSbTS36BRGyTwqdEkrt0wso+bgMf66cU/QUUQSToUuaeWqEefSvyCPX7++NegoIgmnQpe0khUOMefSgbz+3j7e0R2NJMOo0CXtzBpXSk5WiN8s3Rp0FJGEUqFL2umVn8OMC/vz3KoaDjVornTJHCp0SUu3TCzj6PFmnqmsDjqKSMKo0CUtnT+ggItKe/LEsm2ahVEyhgpd0tbNEwayee/7moVRMoYKXdLW9Av6cU7XbH67bFvQUUQSQoUuaSsvO8wNFSW8uGEPu+sbgo4jEncqdElrN11SSos7T76hG0lL+lOhS1ob2Dufy4cWMX/Fdk40twQdRySuVOiS9m6eMJA9hxr5ywbN7yLpTYUuae/K4X0Y0LML85Zrt4ukNxW6pL1wyLhxXAmvVe1l6973g44jEjcqdMkIN44rIRwyHRyVtKZCl4xwbo88rhrRh2dWVtPY1Bx0HJG4iKrQzWyqmW0ysyozu/cUY24wsw1mtt7M5sU2pkjn3XTJQPa/f5xF63VwVNJTh4VuZmHgYWAaMBKYbWYjTxozFLgPmOTuo4B/jX1Ukc6ZPKSQkl5dmLdcV45KeopmC308UOXum939ODAfmHnSmM8DD7v7AQB3r41tTJHOC4WM2eNLWbZ5P1W1R4KOIxJz0RT6AKDtbdSrI6+1NQwYZmZLzGyZmU1t743M7DYzqzSzyrq6urNLLNIJ119cQnZYB0clPcXqoGgWMBS4ApgNPGJmPU8e5O5z3b3C3SuKiopi9NEi0SvqnsvVI/vy3KpqGk7o4Kikl2gKvQYoafO8OPJaW9XAAnc/4e5bgHdoLXiRpDN7fCkHjp5g0frdQUcRialoCn0FMNTMBplZDjALWHDSmOdp3TrHzApp3QWzOXYxRWJn4uDelPTqwvw3dnQ8WCSFdFjo7t4E3AUsAjYCT7v7ejN7wMxmRIYtAvaZ2QZgMfAVd9ddBSQphULGrHGlLN28j811Ojgq6SOqfejuvtDdh7n7YHf/TuS1b7j7gshjd/e73X2ku1/g7vPjGVqks66/uJhwyHhqhbbSJX3oSlHJSH105aikIRW6ZKxZ40vZ//5xTasraUOFLhnr8qFFDOipg6OSPlTokrHCIeOGitZpdbfvOxp0HJFOU6FLRru+opiQwdOV2kqX1KdCl4zWv2cXpgwr4pmVO2jSPUclxanQJePNGl/KnkONLN6k+YUktanQJeN9YHgfirrnMl8TdkmKU6FLxssOh7ju4mIWb6pld31D0HFEzpoKXQSYNa6EFodndHBUUpgKXQQY2DufiYN781TlDlpaPOg4ImdFhS4SceO4EqoPHGPJe3uDjiJyVlToIhHXjOpLz67ZzNeEXZKiVOgiEXnZYT520QBeXL+b/e8fDzqOyBlToYu0MWtcKSeanedWVQcdReSMqdBF2jivb3cuKu3J/BU7cNfBUUktKnSRk8waV0JV7RFWbjsQdBSRM6JCFznJtaP7k58T5klNqyspRoUucpL83CxmjOnPC2t3cqjhRNBxRKKmQhdpx6xxpTScaOEPb+4MOopI1FToIu0YXVzAiH49eGqFJuyS1KFCF2mHmTFrXAnrag6xrqY+6DgiUVGhi5zCR8cMIDcrxJOaVldShApd5BQKumbz4Qv6seDNnRw93hR0HJEOqdBFTmPW+FIONzbxwppdQUcR6ZAKXeQ0xpWdQ3lRvibskpSgQhc5DTNj9rhSVm47wDt7DgcdR+S0VOgiHfjExcXkhHVwVJKfCl2kA73yc7h61Lk8t6qGhhPNQccROSUVukgUbhpfSv2xE/x53e6go4ickgpdJAoTyntT1rsr87TbRZJYVIVuZlPNbJOZVZnZvacZ9wkzczOriF1EkeCFQsaN40p5Y8t+3qs7EnQckXZ1WOhmFgYeBqYBI4HZZjaynXHdgX8Blsc6pEgyuO7iYrJCxnxtpUuSimYLfTxQ5e6b3f04MB+Y2c64bwPfAxpimE8kaRR1z+XqUefy7MpqGpt0cFSSTzSFPgBoe1VFdeS1/2VmY4ESd3/hdG9kZreZWaWZVdbV1Z1xWJGgzR5fyoGjOjgqyanTB0XNLAT8ELino7HuPtfdK9y9oqioqLMfLZJwkwYXUtqrK08s124XST7RFHoNUNLmeXHktb/rDpwP/M3MtgITgAU6MCrpKBQyZo9vPThaVasrRyW5RFPoK4ChZjbIzHKAWcCCvy9093p3L3T3MncvA5YBM9y9Mi6JRQJ2fUUx2WFj3nLN7yLJpcNCd/cm4C5gEbAReNrd15vZA2Y2I94BRZJNYbdcrhnVl2dX7tCVo5JUotqH7u4L3X2Yuw929+9EXvuGuy9oZ+wV2jqXdHfTJaUcatC0upJcdKWoyFm4tLw35YX5PLF8W9BRRP6XCl3kLJgZN11SyqrtB9mw81DQcUQAFbrIWbvu4mJys0L8VlvpkiRU6CJnqWfXHD5yYX+eX13D4YYTQccRUaGLdMbNEwZy9Hgzv19d0/FgkThToYt0woUlPRldXMDjS7fh7kHHkQynQhfppDkTBvJu7RHe2LI/6CiS4VToIp30kdH96ZGXxePLdHBUgqVCF+mkLjlhrq8o4c/rdlN7SLNHS3BU6CIxMGfCQJpaXLMwSqBU6CIxMKgwnyvOK2LeG9s53tQSdBzJUCp0kRj59MQy6g438qd1mt9FgqFCF4mRKUOLGFSYz69e3xp0FMlQKnSRGAmFjJsnDGT19oOsqT4YdBzJQCp0kRi6rqKYrjlhfv26TmGUxFOhi8RQj7xsPj52AP/91k72HmkMOo5kGBW6SIzdMnEQx5tbeGKZTmGUxFKhi8TYkD7duOK8Ih5fto3GJt2iThJHhS4SB7deNoi9RxpZ8ObOoKNIBlGhi8TBZUMKOe/c7jz62hbNwigJo0IXiQMz47OXlfH27sMs3bwv6DiSIVToInEyc8wAeuXn8NhrW4KOIhlChS4SJ3nZYeZcUspLb9eyue5I0HEkA6jQReLo5kvLyA6HeORVbaVL/KnQReKoqHsu111czO9WVVN7WHOlS3yp0EXi7POTyznR3MKvNWmXxJkKXSTOBhXmM3VUXx5fuo0jjU1Bx5E0pkIXSYDbpwzmUEMT89/QdAASPyp0kQQYU9KTCeW9ePS1LbqjkcSNCl0kQW6fMphd9Q08/2ZN0FEkTUVV6GY21cw2mVmVmd3bzvK7zWyDma0xs5fMbGDso4qktiuGFTGqfw9+triK5hZNByCx12Ghm1kYeBiYBowEZpvZyJOGrQYq3H008Czw/VgHFUl1ZsaXPjCErfuO8sc1mrRLYi+aLfTxQJW7b3b348B8YGbbAe6+2N2PRp4uA4pjG1MkPVw9si/Dzu3GQy9X0aKtdImxaAp9ALCjzfPqyGuncivwp/YWmNltZlZpZpV1dXXRpxRJE6GQ8cUrh/Bu7REWrd8ddBxJMzE9KGpmc4AK4MH2lrv7XHevcPeKoqKiWH60SMq4dnR/ygvz+enLVZpaV2IqmkKvAUraPC+OvPYPzOwq4H5ghrvrZooipxAOGXdeOYQNuw7x1421QceRNBJNoa8AhprZIDPLAWYBC9oOMLOLgF/QWub6DRXpwMwx/Snr3ZUfvLhJ+9IlZjosdHdvAu4CFgEbgafdfb2ZPWBmMyLDHgS6Ac+Y2ZtmtuAUbyciQHY4xL9eNYy3dx/mhbW7go4jacKC2odXUVHhlZWVgXy2SDJobnGm/fgVmpqdF//tcrLCus5POmZmK929or1l+g0SCUg4ZNz9ofPYvPd9nlutq0el81ToIgG6ZtS5XDCggB//9V0am5qDjiMpToUuEiAz456rh1Fz8BhPLtdMjNI5KnSRgE0ZVsSl5b358UvvUn/sRNBxJIWp0EUCZmbc/+ERHDx2gp8trgo6jqQwFbpIEjh/QAEfv6iYXy7Zyo79Rzv+BpF2qNBFksRXrjmPUAi+9+e3g44iKUqFLpIk+hbkcdvkcv64Zherth8IOo6kIBW6SBK5fcpg+nTP5Zt/WK+bYMgZU6GLJJH83Cy+fu1I1tbU88TybUHHkRSjQhdJMteO7sdlQwp5cNEmag83BB1HUogKXSTJmBkPzBxF44kWvrtQB0gleip0kSRUXtSN26eU8/vVNbz+3t6g40iKUKGLJKkvXjmE0l5due+5tRw93hR0HEkBKnSRJJWXHeb7141m+/6j/MeftOtFOqZCF0liE8p785mJg/jN0m0sqdKuFzk9FbpIkvvq1PMoL8znq8+u4XCDJu+SU1OhiyS5vOww//+GC9lVf4xvLlgfdBxJYip0kRQwtvQc7vrAUJ5bVcPTK3YEHUeSlApdJEX8yweHMmlIb77+h3Vs2Hko6DiShFToIikiHDJ+POsienbN5s4nVnJI+9PlJCp0kRRS2C2Xh24ay44Dx7jn6bc0gZf8AxW6SIoZV9aLr394BH/ZsIcH/ns97ip1aZUVdAAROXO3TBpEzcFjPPLqFvr37MLtUwYHHUmSgApdJEXdN20EO+sb+O6f3qZvQR4zxwwIOpIETIUukqJCIeMH11/I3sON/NtTb9Lc4nx8bHHQsSRA2ocuksLyssM8dss4Lh3cm7uffovHl+mmGJlMhS6S4vJzs3j00+O4akQfvv78Oh5eXKUDpRlKhS6SBvKyw/x8zsXMHNOfBxdt4o7frqT+mM5TzzQqdJE0kR0O8Z83juFrHx7BSxtrufanr7K2uj7oWJJAKnSRNGJmfG5yOU/dPoGmZuejP1vC155fy94jjUFHkwSIqtDNbKqZbTKzKjO7t53luWb2VGT5cjMri3lSEYnaxQN7sfDLk5lzSSlPvrGDKx/8Gw+9/C61h3TT6XRmHR08MbMw8A7wIaAaWAHMdvcNbcbcCYx29zvMbBbwMXe/8XTvW1FR4ZWVlZ3NLyIdqKo9wncXbuSlt2sJGVw2tIhrR/djVP8eDC7qRl52OOiIcgbMbKW7V7S7LIpCvxT4lrtfE3l+H4C7f7fNmEWRMUvNLAvYDRT5ad5chS6SWFW1R/j96mqeX72TmoPHADCD/gVdyMkKYQYhMyzgnJngyx8cykcu7H9W33u6Qo/mwqIBQNsJmKuBS041xt2bzKwe6A38wz2zzOw24DaA0tLSqMKLSGwM6dONr1wznHs+dB7v1B6mqvYI7+45wo4DR2lqdlq89Uvir6BLdlzeN6FXirr7XGAutG6hJ/KzRaRVKGQM79uD4X17BB1FYiyag6I1QEmb58WR19odE9nlUgDsi0VAERGJTjSFvgIYamaDzCwHmAUsOGnMAuDTkcfXAS+fbv+5iIjEXoe7XCL7xO8CFgFh4DF3X29mDwCV7r4AeBR43MyqgP20lr6IiCRQVPvQ3X0hsPCk177R5nEDcH1so4mIyJnQlaIiImlChS4ikiZU6CIiaUKFLiKSJjq89D9uH2xWB5zt7VUKOekq1Aygdc4MWufM0Jl1HujuRe0tCKzQO8PMKk81l0G60jpnBq1zZojXOmuXi4hImlChi4ikiVQt9LlBBwiA1jkzaJ0zQ1zWOSX3oYuIyD9L1S10ERE5iQpdRCRNJHWhZ+LNqaNY57vNbIOZrTGzl8xsYBA5Y6mjdW4z7hNm5maW8qe4RbPOZnZD5Ge93szmJTpjrEXxu11qZovNbHXk93t6EDljxcweM7NaM1t3iuVmZj+J/PdYY2ZjO/2h7p6UX7RO1fseUA7kAG8BI08acyfwX5HHs4Cngs6dgHW+EugaefyFTFjnyLjuwCvAMqAi6NwJ+DkPBVYD50Se9wk6dwLWeS7whcjjkcDWoHN3cp0vB8YC606xfDrwJ8CACcDyzn5mMm+hjweq3H2zux8H5gMzTxozE/h15PGzwAfNLJXvcdvhOrv7Ync/Gnm6jNY7SKWyaH7OAN8Gvgc0JDJcnESzzp8HHnb3AwDuXpvgjLEWzTo78Pf74hUAOxOYL+bc/RVa7w9xKjOB33irZUBPM+vXmc9M5kJv7+bUA041xt2bgL/fnDpVRbPObd1K6//hU1mH6xz5p2iJu7+QyGBxFM3PeRgwzMyWmNkyM5uasHTxEc06fwuYY2bVtN5/4UuJiRaYM/1771BCbxItsWNmc4AKYErQWeLJzELAD4FbAo6SaFm07na5gtZ/hb1iZhe4+8EgQ8XZbOBX7v4DM7uU1rugne/uLUEHSxXJvIWeiTenjmadMbOrgPuBGe7emKBs8dLROncHzgf+ZmZbad3XuCDFD4xG83OuBha4+wl33wK8Q2vBp6po1vlW4GkAd18K5NE6iVW6iurv/Uwkc6Fn4s2pO1xnM7sI+AWtZZ7q+1Whg3V293p3L3T3Mncvo/W4wQx3rwwmbkxE87v9PK1b55hZIa27YDYnMGOsRbPO24EPApjZCFoLvS6hKRNrAfCpyNkuE4B6d9/VqXcM+khwB0eJp9O6ZfIecH/ktQdo/YOG1h/4M0AV8AZQHnTmBKzzX4E9wJuRrwVBZ473Op809m+k+FkuUf6cjdZdTRuAtcCsoDMnYJ1HAktoPQPmTeDqoDN3cn2fBHYBJ2j9F9etwB3AHW1+xg9H/nusjcXvtS79FxFJE8m8y0VERM6ACl1EJE2o0EVE0oQKXUQkTajQRUTShApdRCRNqNBFRNLE/wBJWaabb7ND9wAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "p = torch.linspace(0.,1,100)\n",
    "f = combine_scheds([0.3, 0.4, 0.3], [SchedLin(1.,1.), SchedCos(1.,0.), SchedLin(0.,.0), ])\n",
    "plt.plot(p, [f(o) for o in p]);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAp4klEQVR4nO3dd3xUZb7H8c9v0ntIIaGFUBKa9FBFRUHFfhWl2BUX0dUt6q7u1et63esWy7q6q65YVgUbYIuK4lIUpSdSExIIkEBCGoQ0Qvpz/0h2N8uCGZLJnCm/9+vF6zXlMPM9TPhyeOac5xFjDEoppdyfzeoASimlHEMLXSmlPIQWulJKeQgtdKWU8hBa6Eop5SF8rXrjmJgYk5iYaNXbK6WUW0pPTz9ijIk91XOWFXpiYiJpaWlWvb1SSrklEck73XM65KKUUh5CC10ppTyEFrpSSnkILXSllPIQWuhKKeUh2i10EXldREpEZNdpnhcReV5EckRkh4iMcXxMpZRS7bHnCP0NYMYPPH8JkNT6az7wUudjKaWUOlPtnodujFkrIok/sMlVwFumZR7ejSISKSI9jDGFjgqpnKuhqZnv845RUlXH0eo6Kk40EuhnIzTQl9AAX+LDA+kZGUR8RCB+Pjpqp5SrcMSFRb2AQ23u57c+9h+FLiLzaTmKJyEhwQFvrRyp7Hg9724+yKINeRRV1ra7vU2gX0wIQ3qEM6RHOGMSujE6IZJAPx8npFVKncypV4oaYxYCCwFSUlJ0ZQ0XYYzh9XW5PPllFnWNzUwZGMOvrxjKgO6hRIf4ExHkR11jM8frGqmsbaCooo7D5Sc4dKyG7KIqtueX89mOln+//X1tjO4TybnJsVw8LJ6B3UMt3julvIcjCr0A6NPmfu/Wx5QbOF7XyIMf7OCzHYVMHxLHgzMGkRQX9h/b+frYCAnwpXt4IAO7/+fzFTUNpOWVsXH/UdbvO8pTK7J5akU2A2JDuGxET64b25s+UcHO2CWlvJYjCj0VuEdE3gMmABU6fu4eDpXVMO/NLeSUVPPgjMEsOK8/ItKh14oI9mPakDimDYkD4HD5CVbuLubLXUX8efVenl+1l4n9o5g7PoFLh/fQsXeluoC0t6aoiLwLTAVigGLg14AfgDHmr9LSAH+h5UyYGuA2Y0y7s26lpKQYnZzLOker65j50nqO1TTw4g1jOHtgTJe9V0H5CT5Mz2dpej4Hy2qIDw/k5sl9uX58ApHB/l32vkp5IhFJN8aknPI5qxaJ1kK3Tk19I3Nf2URWYSXv/GgiY/t2c8r7Njcbvt5TwmvfHWBdzlFCA3y57exE7pjSn4hgP6dkUMrd/VChWzZ9rrJGY1Mz976zlZ355fz1xrFOK3MAm024YHAcFwyOI/NwJX9Zs5c/r87hjXW53HFOf350bj+C/fVHUqmO0oFML/PUV9msyirhf68cxkXD4i3LMbRnOC/eMJblPzmHSQOieXblHi54+hs+SM+nuVlPgFKqI7TQvcjmA2UsXLufueMTuGlSotVxgJZiX3hzCksXTKJ7eAD3L93O1S+uY1dBhdXRlHI7Wuhe4nhdIw8s3U6fbsE8ctkQq+P8h3GJUXx899n8cdZICsprufIv3/F/n2VyvK7R6mhKuQ0tdC/xxPLdHDpWw9PXjSQkwDXHqW024ZoxvVl1/3nMGZ/Aq98d4KJn17I+54jV0ZRyC1roXuDr7BLe2XSQ+ef0Z3y/KKvjtCsiyI/fXj2cZQsmEeBr4/pXN/FYagYn6pusjqaUS9NC93D1jc38OjWDgd1D+fmFyVbHOSMpiVF8/pNzuHVyIm+sz+WyP39L5uFKq2Mp5bK00D3cWxtyyTtaw/9cPtQtJ80K8vfhsSuH8fYdEzhe18h/vbiOxRvzsOr6CaVcmRa6Bzt2vJ7nV+3lvORYzkuOtTpOp5w9MIblPzmHyQOieeTjXdzzzlaqahusjqWUS9FC92DPrdpLdV0jD7vgWS0dER0awOu3jOOhSwbzZUYRV7+4nn2l1VbHUsplaKF7qH2l1SzemMfc8Qkkn2L2RHdlswkLzhvA4nkTKDtez3/9ZR2rdhdbHUspl6CF7qGeXpFNoJ+P230Raq9JA6L59N4p9I0JZt6babz09T4dV1deTwvdA+0pruKLXUXcPqUfMaEBVsfpMr0ig1i2YDJXjOzJH77M4qEPdtLQ1Gx1LKUs45pXmKhOeWFNDsH+Ptw2OdHqKF0u0M+H52aPol90MM+vzuHQsRpeunEsEUE6e6PyPnqE7mFyjxzn0+2HuWliX7qFeMdc4zabcN9Fg3jmupFsyS1j9ssbKLZjTVSlPI0Wuod56et9+PrYmHdOP6ujON3Msb35263jOVRWwzV6BozyQlroHqSg/AQfbs1n7rg+dA8LtDqOJaYkxfDe/EnUNjRx7Uvr2ZFfbnUkpZxGC92DvLJ2P8bA/PMGWB3FUsN7R7DsrsmEBPhy/Sub2JJbZnUkpZxCC91DVNY2sCTtEFeN6kWvyCCr41iuX0xIyxzrYQHc/NpmvturMzYqz6eF7iGWpuVTU9/EbWcnWh3FZfSICOL9OyfRNzqY29/cwpqsEqsjKdWltNA9QHOz4a0NuYzt242zekVYHcelxIYF8O6PJjIoLow7F6VrqSuPpoXuAb7eU0Le0Rpu9YLzzjuiW4g/i+dNIDk+lDsXpfN1tpa68kxa6B7gjfV5xIUHMOMs6xZ9dnURwX4snjeBpLhQ5i9KZ+2eUqsjKeVwWuhubl9pNWv3lHLDhL74+ejH+UMig/15+44JDIgNZf6iNDbtP2p1JKUcShvAzS3akIe/j4254xOsjuIWIoP9WTRvPL0ig5j3ZhrbD5VbHUkph9FCd2Mn6pv4ID2fS4fHExvmuZNwOVpMaACL75hAZLAfN7++mawiXdZOeQYtdDf2ZUYhVXWNzB6nR+dnqkdEEO/cMZFAPxs3vbaZQ2U1VkdSqtO00N3Y+1sO0Tc6mIn9o6yO4pYSooNZNG8C9Y3N3PTaJo5U11kdSalO0UJ3U3lHj7NxfxmzUvogIlbHcVvJcWG8fmsKRZW13Pa3LVTXNVodSakO00J3U0vSDmETmDmmt9VR3N7YvlG8eMMYMgsruWtxui6SodyWFrobamo2LEvP57zkWOIjvHNWRUe7YHAcv79mON/uPcJDH+zU5eyUW7Kr0EVkhohki0iOiDx0iucTRGSNiGwVkR0icqnjo6p/WLunlOLKOmaP62N1FI9yXUoffjY9iQ++z+fZlXutjqPUGWt3CToR8QFeAC4E8oEtIpJqjMlss9kjwBJjzEsiMhRYDiR2QV5Fy5eh0SH+XDA4zuooHuen05I4XH6C51ftpXdkELP0H03lRuw5Qh8P5Bhj9htj6oH3gKtO2sYA4a23I4DDjouo2iqvqWdVVjH/NboX/r46YuZoIsITVw/nnKQY/vujnazL0Wl3lfuwpxF6AYfa3M9vfaytx4AbRSSflqPze0/1QiIyX0TSRCSttFTn0uiIz3cW0tBkuHr0yR+BchQ/Hxsv3DCGfjEh3LU4nZwSXcpOuQdHHeLNBd4wxvQGLgUWich/vLYxZqExJsUYkxIbG+ugt/Yun2w9zIDYEIb1DG9/Y9Vh4YF+vH7rOPx8bMx7cwtlx+utjqRUu+wp9AKg7UBi79bH2poHLAEwxmwAAoEYRwRU/5J/rIbNuWVcPbqXnnvuBH2igll4cwqFFbUsWJxOfaOezqhcmz2FvgVIEpF+IuIPzAFST9rmIDANQESG0FLoOqbiYKnbW76auGqUDrc4y9i+3Xjq2hFsPlDGr1Mz9HRG5dLaPcvFGNMoIvcAKwAf4HVjTIaIPA6kGWNSgfuBV0Tk57R8QXqr0Z98hzLG8PHWAsb27UafqGCr43iVq0b1Iruoihe/3seQHmHcPCnR6khKnVK7hQ5gjFlOy5edbR97tM3tTOBsx0ZTbe0urGJPcTW/uWqY1VG80gMXDWJPcRX/+2kmA2NDmTxQRxSV69Hz3tzEJ9sK8LUJl43oaXUUr2SzCc/OHsWA2BDufud7nZ1RuSQtdDfQ3GxI3X6Yc5NjiQrxtzqO1woL9OOVm1NobjbMX5TOifomqyMp9W+00N3A9wePUVhRy5Uj9ejcan2jQ3h+7miyiip58IMd+iWpcila6G7g852F+PvamDaku9VRFDB1UHceuGgQqdsP8+q3B6yOo9Q/aaG7uOZmw/KdhZyXHEtYoJ/VcVSru6cO4JKz4vndF7tZv0+nB1CuQQvdxX1/8BjFlXVcPqKH1VFUGyLCU9eNpF9MCD95dytFFbVWR1JKC93V/Wu4RWdWdDWhAb68fNNYTtQ3cffbeiWpsp4WugtrO9wSGmDXJQPKyQZ2D+PJa0fy/cFyfrt8t9VxlJfTQndhOtziHi4b0YN5U/rxxvpcPt2uM0cr62ihu7DPduhwi7t46JLBjO3bjYc+2MG+Up1uV1lDC91FNTcbvthVyFQdbnELfj42/nL9aAL8fLh78fd60ZGyhBa6i9p6qJziyjouHa7DLe6iR0QQf5o9ij0lVTzy8S696Eg5nRa6i/oqowg/H+H8wXoxkTs5NzmWey9oWWh6aXq+1XGUl9FCd0HGGFZkFDFpQAwRQXoxkbv56bQkJvWP5tFPdrG3uMrqOMqLaKG7oOziKnKP1nDxMP0y1B352ITn5owiNMCXH7+j4+nKebTQXdCKXcWIwIVDtdDdVffwQJ6dPYq9JdX8OnWX1XGUl9BCd0ErMooYm9CN7mGBVkdRnXBOUiw/njqQJWn5fLz15GV4lXI8LXQXc6ishszCSi4eFm91FOUAP5uexLjEbjz80U5yjxy3Oo7ycFroLmZFRhGAFrqH8PWx8ac5o/H1sfGT97bqfC+qS2mhu5gVGUUMjg8jIVoXgvYUvSKDePLaEezIr+CpFVlWx1EeTAvdhRypriMt75genXugi4fFc9PEvrzy7QHWZJdYHUd5KC10F7J6dwnG6HCLp3r4siEMjg/jF0u3U1pVZ3Uc5YG00F3IV5nF9IoMYkiPMKujqC4Q6OfD83NHU1XbyANLt9PcrFMDKMfSQncRJ+qb+C6nlOlDuiMiVsdRXSQ5LoxHLh/KN3tKeX2drkeqHEsL3UWsyzlCbUMzFw7V4RZPd+OEBC4cGscfvsxiV0GF1XGUB9FCdxF/zywmLMCX8f2irI6iupiI8IeZI4gK8edn72/TqQGUw2ihu4DmZsOqrGLOGxSLv69+JN4gKsSfZ64bRU5JtS5dpxxG28MFbD1UzpHqep27xctMSYrhR+f0Y9HGPFbtLrY6jvIAWuguYOXuYnxtwtRknfvc2zxw8SCG9Ajnl8t2UFJVa3Uc5ea00F3AysxixveLIiJY5z73NgG+Pjw/ZxTVdY08uGyHrnKkOsWuQheRGSKSLSI5IvLQabaZJSKZIpIhIu84Nqbnyj1ynL0l1Trc4sWS4sJ4+LIhrMkuZfHGPKvjKDfWbqGLiA/wAnAJMBSYKyJDT9omCfgVcLYxZhjwM8dH9UwrW8dOpw3WQvdmN03sy3nJsfzf57vJKdFVjlTH2HOEPh7IMcbsN8bUA+8BV520zY+AF4wxxwCMMTpZhZ1WZ5WQ1D1UJ+PyciLCU9eOINjfh5+9v01nZVQdYk+h9wIOtbmf3/pYW8lAsoisE5GNIjLjVC8kIvNFJE1E0kpLSzuW2INU1jaw+UAZ04bo0blqWeXod9eMYFdBJX9aucfqOMoNOepLUV8gCZgKzAVeEZHIkzcyxiw0xqQYY1JiY2Md9Nbua+2eUhqbDdOG6NktqsWMs+KZldKbv36zjy25ZVbHUW7GnkIvAPq0ud+79bG28oFUY0yDMeYAsIeWglc/YPXuEiKD/RiT0M3qKMqFPHrFMHp1C+K+Jduormu0Oo5yI/YU+hYgSUT6iYg/MAdIPWmbj2k5OkdEYmgZgtnvuJiep6nZsCa7hPMHdcfHppNxqX8JDfDl2VmjKDh2gsc/zbA6jnIj7Ra6MaYRuAdYAewGlhhjMkTkcRG5snWzFcBREckE1gC/MMYc7arQnmDrwWMcq2nQ4RZ1SimJUdw1dQBL0vL/uSyhUu3xtWcjY8xyYPlJjz3a5rYB7mv9peywcncJvjbh3GT9LkGd2k+nJfN1dim/+nAnYxK6ERsWYHUk5eL0SlGLrM5quTo0PFCvDlWn5u9r40+zW64i/dWHehWpap8WugUOldWwp7iaCwbrcIv6YUlxYfzy4kGs3F3CkrRD7f8G5dW00C3wj5n19PxzZY/bz+7HpP7RPP5pJgeP1lgdR7kwLXQLrM4upX9MCP1iQqyOotyAzSY8PWskNptw/9JtNOlapOo0tNCd7HhdIxv3HdXhFnVGekUG8b9XDmNL7jFe/VbPCFanpoXuZOtyjlDf1KyFrs7Y1aN7MWNYPM98tYfdhZVWx1EuSAvdydZklxAW4EtKoq4dqs6MiPDE1WcRHuTLz9/fRl2jrkWq/p0WuhMZY1idVcI5yTG6dqjqkOjQAH53zQiyiqr408q9VsdRLkZbxYkyDldSXFnHBTr3ueqEC4fGMSulNy9/s4/0PJ3AS/2LFroTrc4qQQSmDtKrQ1Xn/M/lQ+kREcR9S7ZTU68TeKkWWuhOtDqrhJG9I4kJ1Uu4VeeEBfrxzKyRHCyr4XfLs6yOo1yEFrqTHKmuY3t+uZ7dohxmYv9obj+7H4s25vHNHl0wRmmhO83X2aUYgxa6cqhfXDyIgd1D+eWy7VTUNFgdR1lMC91J1mSVEBcewLCe4VZHUR4k0M+HZ2eN4mh1PY+m7rI6jrKYFroTNDQ1s3ZPKecP6o6ILmahHGt47wjuvSCJT7Yd5vMdhVbHURbSQneCLbllVNU16nCL6jJ3nz+Akb0jeOTjnZRU1lodR1lEC90JVu8uwd/HxtkDY6yOojyUn4+NZ2aNoqa+iYc+3Klzp3spLXQnWJ1dwoT+UYQE2LVAlFIdMrB7KA/OGMzqLJ073VtpoXex3CPH2V96nGk63KKc4NbJif+cO/1Qmc6d7m200LvY6qwSAL3cXznFP+dOF+H+Jdt17nQvo4XexdZklzAgNoSE6GCroygv0SsyiF9fOYzNuWW8/t0Bq+MoJ9JC70LVdY1s2l+mS80pp5s5phcXDY3jqRXZZBdVWR1HOYkWehf6bm/LYhbnD9Lxc+VcIsJvrxn+z7nT6xubrY6knEALvQutziomLNCXlMRuVkdRXigmNIDfXj2czMJKnlu1x+o4ygm00LtIc7NhdVYp5yXH4uejf8zKGhcNi+e6sb156et9pOcdszqO6mLaNF1kZ0EFR6rrmDZEh1uUtR69omXu9PuXbNO50z2cFnoXWZVVgk1garIWurLWP+ZOzyur4YnPd1sdR3UhLfQusjqrmLF9u9EtxN/qKEoxsX80PzqnP29vOsia7BKr46guooXeBYoqatlVUKkXEymXct+FyQyKC+OXy3ZQdrze6jiqC2ihd4F/HAHp+LlyJYF+Pjw7exTlNfU8/JFO4OWJtNC7wKrdJfTuFkRS91Croyj1b4b2DOe+Cwfxxa4iPvy+wOo4ysHsKnQRmSEi2SKSIyIP/cB2M0XEiEiK4yK6l9qGJtblHGHaYF3MQrmm+ef2Z3xiFL9OzdAJvDxMu4UuIj7AC8AlwFBgrogMPcV2YcBPgU2ODulONuw7yomGJi7Qy/2Vi/KxCc/MGgnA/Ut1Ai9PYs8R+nggxxiz3xhTD7wHXHWK7X4D/AHw6uVSVmUVE+zvw8T+UVZHUeq0+kQF89iVw9h8oIxXvt1vdRzlIPYUei+g7Wz5+a2P/ZOIjAH6GGM+/6EXEpH5IpImImmlpaVnHNbVGWNYmVnCuUmxBPj6WB1HqR80c0wvLjkrnme+yibjcIXVcZQDdPpLURGxAX8E7m9vW2PMQmNMijEmJTY2trNv7XIyDldSVFnL9KE63KJcn4jw26uH0y3Yn5++t43ahiarI6lOsqfQC4A+be73bn3sH8KAs4CvRSQXmAikeuMXo3/PLEYEzh/kef9YKc/ULcSfp68bSU5JNb//IsvqOKqT7Cn0LUCSiPQTEX9gDpD6jyeNMRXGmBhjTKIxJhHYCFxpjEnrksQubFVWMWMTuhEdGmB1FKXsdm5yLLedncgb63P5Wq8idWvtFroxphG4B1gB7AaWGGMyRORxEbmyqwO6i8KKE+wqqNThFuWWHpwxmOS4UB5YuoOj1XVWx1EdZNcYujFmuTEm2RgzwBjzROtjjxpjUk+x7VRvPDpfubvlyGa6Xh2q3FCgnw/PzRlN5YkGHvxgh15F6qb0SlEHWZlZTGJ0MANi9epQ5Z6G9AjnwUsGs3J3CYs3HbQ6juoALXQHOF7XyIZ9R5k+JE6vDlVu7bbJiZybHMv/fZbJ3mJdi9TdaKE7wLd7S6lvatbFoJXbs9mEp68dQUiALz95bxt1jXoqozvRQneAv2eWEBHkp2uHKo/QPTyQp64dwe7CSp78MtvqOOoMaKF3UmNTM6uyipk2uLuuHao8xrQhcdwyqS+vfXdAF8RwI9pAnbQ5t4zymgYuGqbDLcqz/OrSIQyOD+OBJdspqfLqKZrchhZ6J32VUUyAr41zk/XqUOVZAv18+PPc0VTXNXL/ku0066yMLk8LvROMMfw9s5hzkmIJ9ve1Oo5SDpcUF8ajVwzl271HdFZGN6CF3gkZhyspKD+hwy3Ko10/PoFLzornqRXZbDtUbnUc9QO00Dvhq4wibALT9XRF5cFEhN9fM4K48EDuffd7KmsbrI6kTkMLvRO+yixmXGIUUSH+VkdRqktFBPvx/NzRHC6v5Vcf6ALTrkoLvYPyjh4nq6iKi4bFWx1FKacY27cb91+UzOc7C3l386H2f4NyOi30DvoqoxiAi3R2ReVFFpw7gHOSYnjs0wwyD1daHUedRAu9g77MKGJoj3D6RAVbHUUpp7HZhGdnjyIyyI973vme6rpGqyOpNrTQO6Coopb0vGNcOlyHW5T3iQkN4Pm5o8k9epz//lDH012JFnoHfLmrEIBLh/ewOIlS1pjYP5r7LkwmdfthHU93IVroHbB8VxGD48Por3OfKy9299SBnJscy2OfZrCroMLqOAot9DNWUlXLltwyLjlLj86Vd7PZhD/NHkV0iD93vZ1ORY2en241LfQztCKjGGPQ8XOlgKgQf164YQyF5bXcv3SbzvdiMS30M/TFzkIGdg8lKS7M6ihKuYQxCd14+LIhrNxdwstrdb4XK2mhn4Gj1XVs3H+US8/So3Ol2rp1ciKXjejBUyuyWJdzxOo4XksL/Qx8lVlMs4FL9OwWpf6NiPDkzBH0jw3l3ne3UlB+wupIXkkL/Qx8vqOQxOhgBsfrcItSJwsJ8OXlm8ZS39jM3YvTqW3Q9UidTQvdTiVVtazfd4QrRvZERKyOo5RLGhAbytPXjWR7fgWPpWZYHcfraKHbafmOQpoNXDmyp9VRlHJpM86K5+6pA3hvyyEWb8yzOo5X0UK3U+r2wwyOD9OzW5Syw/0XDWLqoFgeS81g84Eyq+N4DS10Oxwqq+H7g+VcNaqX1VGUcgs+NuG5OaPpExXM3W+nU1ihX5I6gxa6HVK3HwbgipF6dotS9ooI8uOVm8dS29DM/LfSOVGvX5J2NS10O3y6/TBj+3ajdzedKlepMzGwexh/mj2KXYcr+MWy7TozYxfTQm9HdlEVWUVV+mWoUh00fWgcv7x4MJ/tKOQvq3OsjuPR7Cp0EZkhItkikiMiD53i+ftEJFNEdojIKhHp6/io1kjdXoBNdKpcpTpjwXn9uWZ0L575+x6+2FlodRyP1W6hi4gP8AJwCTAUmCsiQ0/abCuQYowZASwDnnR0UCs0Nxs+2XaYswfGEBsWYHUcpdyWiPDba4YzOiGSny/Zxo78cqsjeSR7jtDHAznGmP3GmHrgPeCqthsYY9YYY2pa724Eejs2pjU2HSgj/9gJrh3rEbujlKUC/XxYeFMK0SEBzHszTacH6AL2FHovoO2SJPmtj53OPOCLUz0hIvNFJE1E0kpLS+1PaZFl6fmEBfhy0VCdjEspR4gNC+CN28ZRW9/E7X/bQlWtzqHuSA79UlREbgRSgKdO9bwxZqExJsUYkxIbG+vIt3a46rpGlu8s5PKRPQjy97E6jlIeIykujJduHMu+0mp+/M5WGpqarY7kMewp9AKgT5v7vVsf+zciMh14GLjSGFPnmHjWWb6zkBMNTVw7tk/7GyulzsiUpBieuPos1u4p1YWmHcjXjm22AEki0o+WIp8DXN92AxEZDbwMzDDGlDg8pQWWpefTPyaEMQmRVkdRyiPNHpdAQXktz6/aS4/IIO67MNnqSG6v3SN0Y0wjcA+wAtgNLDHGZIjI4yJyZetmTwGhwFIR2SYiqV2W2Anyjh5n84EyZo7trTMrKtWFfj49iVkpvXl+1V7e3XzQ6jhuz54jdIwxy4HlJz32aJvb0x2cy1IffF+ACFwzRuduUaoriQhPXD2ckqo6Hv5oJ1Eh/lw8TE9C6Ci9UvQkTc2GD9LzmTIwhh4RQVbHUcrj+fnYePGGMYzsE8m9725l/T5dwq6jtNBPsiarhILyE9wwIcHqKEp5jWB/X/526zgSo4OZ/1Y6O/MrrI7klrTQT7JoYx5x4QFMHxJndRSlvEpksD9v3T6ByGA/bvnbZvYWV1kdye1oobeRd/Q43+wpZe74BHx99I9GKWeLjwhk8bwJ+NiEG17dRO6R41ZHcivaWm28vekgPjZh7ngdblHKKokxIbxzxwQamw03vLqJ/GM17f8mBWih/1NtQxNL0g5x8bA44sIDrY6jlFdLigvjrdvHU1XbwPWvbOKwzvtiFy30Vp/vKKS8poEbJ3jMzL9KubWzekXw5u3jOXa8njkLN+pkXnbQQm+1aGMe/WNDmDQg2uooSqlWoxO6seiOCRyrqWfOwg06/NIOLXRgS24Z2w6Vc8ukRL0yVCkXM6pPJIvnTaC8poE5CzdyqExL/XS00IGXv9lHt2A/rkvRec+VckUj+0Ty9h0TqKpt5Nq/rtdTGk/D6wt9T3EVK3eXcMvkRIL97ZoJQSllgRG9I1ly5ySaDcx6eYNefHQKXl/oC9fuJ8jPh1smJVodRSnVjkHxYSy9cxLB/r5c/8pGNuw7anUkl+LVhV5YcYJPthUwe1wfuoX4Wx1HKWWHxJgQlt01ibiIQG55fTOf7ThsdSSX4dWF/tq3B2g2MG9KP6ujKKXOQI+IIJYtmMTIPhHc885WXvvugNWRXILXFvrR6jre3XyQy0f0oE9UsNVxlFJnKDLYn0XzJjBjWDy/+SyTx1IzaPTy5ey8ttBfWLOPEw1N3HvBQKujKKU6KNDPhxduGMMdU/rxxvpcbn8zjUovXnjaKws9/1gNizfmcd3YPgzsHmZ1HKVUJ/jYhEcuH8rvrxnO+pwjXPPieq+d1MsrC/2Pf9+DCPzswiSroyilHGTO+AQWzZvAkeo6rvjLd6zMLLY6ktN5XaFnFVXy0dYCbp2cqCsSKeVhJg2I5tN7ppAYHcIdb6XxzFfZNDUbq2M5jdcV+lNfZhMW4MtdUwdYHUUp1QX6RAWzdMEkZqX05s+rc7jptU0UV9ZaHcspvKrQ12SXsCqrhAVTBxAZrOedK+WpAv18ePLakTw5cwRbD5ZzyXPfsmq35w/BeE2hV9c18vCHO0nqHqrnnSvlJWaN68On904hPjyQeW+m8T8f7+J4XaPVsbqM1xT60yuyKays5fczRxDg62N1HKWUkwzsHspHP57MHVP6sXhTHjOeW+uxUwZ4RaGn5x3jzQ253DyxL2P7drM6jlLKyQJ8fXjk8qEsuXMSPiLMfWUj//3RTipqPOucdY8v9NqGJh76YAc9wgP5xYzBVsdRSlloXGIUX/z0XOZN6cd7mw9ywTNfszTtEMZ4xpkwHl3oxhge/mgXe0uqeeKa4YQG6PS4Snm7IH8f/ufyoXx67xT6Rgfzi2U7mPnSetLzyqyO1mkeXeivr8vlg+/z+dn0JM4f1N3qOEopFzKsZwTLFkzmyZkjyD92gpkvbWDBonT2lVZbHa3DPPaQ9bu9R3ji80wuHhbHTy7QK0KVUv/JZhNmjevD5SN78Nq3B/jrN/v4KrOIy0f05MfnD2RQvHtNDSJWjR2lpKSYtLS0Lnnt3YWVzFm4kfjwQD64e7IOtSil7HKkuo5Xvz3Aog25HK9v4sKhcdw2OZFJA6JdZr1hEUk3xqSc8jlPK/R1OUe4c1E6oQG+LLlzEgnROjWuUurMHDtez9/WHWDRxjyO1TQwsHsoN03syxUjexJl8WI4XlPon2wr4IGl2+kXE8Ibt42nZ6TO1aKU6rjahiY+21HIm+tz2VlQga9NmDqoO1eN6sl5g2IJD/RzeqYfKnS7xiJEZAbwHOADvGqM+f1JzwcAbwFjgaPAbGNMbmdCn4lDZTX88e97+GhrARP6RbHw5hQigpz/B62U8iyBfj5cO7Y3M8f0YndhFR9vK+CTbQWs3F2Mr00YlxjF+YNjGZcYxbCeEfj7WnueSbtH6CLiA+wBLgTygS3AXGNMZptt7gZGGGMWiMgc4GpjzOwfet3OHqFX1DSQVVTJlxlFvL3xICJw29n9+PmFSXolqFKqyzQ1G74/eIzVWSWs3l1CdnEVAAG+Nkb0jmBQfBjJcWEMjA0lPiKQ7uGBDv0er1NDLiIyCXjMGHNx6/1fARhjftdmmxWt22wQEV+gCIg1P/DiHS3097cc5LmVezlc0TJ7mk1gVkoffjo9SafDVUo5XXFlLel5x0jLPca2Q8fYW1JNVe2/zxcT7O9DsL8vAb42Avxs/Hx6MleM7Nmh9+vskEsv4FCb+/nAhNNtY4xpFJEKIBo4clKQ+cB8gISEBLvCnywmNIDx/aIY3COcwfFhDOsZQWxYQIdeSymlOisuPJBLh/fg0uE9gJYLGkuq6thXUk1xVS3FlXWUVtVxoqGJ2oYm6hqbiQzumiFhp57PZ4xZCCyEliP0jrzGtCFxTBsS59BcSinlKCJCXHggceGBTn9ve0bwC4A+be73bn3slNu0DrlE0PLlqFJKKSexp9C3AEki0k9E/IE5QOpJ26QCt7TevhZY/UPj50oppRyv3SGX1jHxe4AVtJy2+LoxJkNEHgfSjDGpwGvAIhHJAcpoKX2llFJOZNcYujFmObD8pMcebXO7FrjOsdGUUkqdCY+ebVEppbyJFrpSSnkILXSllPIQWuhKKeUhLJttUURKgbwO/vYYTroK1QvoPnsH3Wfv0Jl97muMiT3VE5YVemeISNrp5jLwVLrP3kH32Tt01T7rkItSSnkILXSllPIQ7lroC60OYAHdZ++g++wdumSf3XIMXSml1H9y1yN0pZRSJ9FCV0opD+HShS4iM0QkW0RyROShUzwfICLvtz6/SUQSLYjpUHbs830ikikiO0RklYj0tSKnI7W3z222mykiRkTc/hQ3e/ZZRGa1ftYZIvKOszM6mh0/2wkiskZEtrb+fF9qRU5HEZHXRaRERHad5nkRkedb/zx2iMiYTr+pMcYlf9EyVe8+oD/gD2wHhp60zd3AX1tvzwHetzq3E/b5fCC49fZd3rDPrduFAWuBjUCK1bmd8DknAVuBbq33u1ud2wn7vBC4q/X2UCDX6tyd3OdzgTHArtM8fynwBSDARGBTZ9/TlY/QxwM5xpj9xph64D3gqpO2uQp4s/X2MmCaiIgTMzpau/tsjFljjKlpvbuRlhWk3Jk9nzPAb4A/ALXODNdF7NnnHwEvGGOOARhjSpyc0dHs2WcDhLfejgAOOzGfwxlj1tKyPsTpXAW8ZVpsBCJFpEdn3tOVC/1Ui1P3Ot02xphG4B+LU7sre/a5rXm0/Avvztrd59b/ivYxxnzuzGBdyJ7PORlIFpF1IrJRRGY4LV3XsGefHwNuFJF8WtZfuNc50Sxzpn/f2+XURaKV44jIjUAKcJ7VWbqSiNiAPwK3WhzF2XxpGXaZSsv/wtaKyHBjTLmVobrYXOANY8wzIjKJllXQzjLGNFsdzF248hG6Ny5Obc8+IyLTgYeBK40xdU7K1lXa2+cw4CzgaxHJpWWsMdXNvxi153POB1KNMQ3GmAPAHloK3l3Zs8/zgCUAxpgNQCAtk1h5Krv+vp8JVy50b1ycut19FpHRwMu0lLm7j6tCO/tsjKkwxsQYYxKNMYm0fG9wpTEmzZq4DmHPz/bHtBydIyIxtAzB7HdiRkezZ58PAtMARGQILYVe6tSUzpUK3Nx6tstEoMIYU9ipV7T6m+B2viW+lJYjk33Aw62PPU7LX2ho+cCXAjnAZqC/1ZmdsM8rgWJgW+uvVKszd/U+n7Tt17j5WS52fs5Cy1BTJrATmGN1Zifs81BgHS1nwGwDLrI6cyf3912gEGig5X9c84AFwII2n/ELrX8eOx3xc62X/iullIdw5SEXpZRSZ0ALXSmlPIQWulJKeQgtdKWU8hBa6Eop5SG00JVSykNooSullIf4f400G9vcIF2OAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "p = torch.linspace(0.,1,100)\n",
    "f = combine_scheds([0.3, 0.7], [SchedCos(0.,1.), SchedCos(1.,0.)])\n",
    "plt.plot(p, [f(o) for o in p]);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## ShowGraph"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#export\n",
    "class ShowGraph(Callback):\n",
    "    \"(Modified) Update a graph of training and validation loss\"\n",
    "    order,run_valid=65,False\n",
    "    names = ['train', 'valid']\n",
    "    def __init__(self, plot_metrics:bool=True, final_losses:bool=False):\n",
    "        store_attr(\"plot_metrics,final_losses\")\n",
    "\n",
    "\n",
    "    def before_fit(self):\n",
    "        self.run = not hasattr(self.learn, 'lr_finder') and not hasattr(self, \"gather_preds\")\n",
    "        if not(self.run): return\n",
    "        self.nb_batches = []\n",
    "\n",
    "    def after_train(self): self.nb_batches.append(self.train_iter)\n",
    "\n",
    "    def after_epoch(self):\n",
    "        \"Plot validation loss in the pbar graph\"\n",
    "        if not self.nb_batches: return\n",
    "        rec = self.learn.recorder\n",
    "        if self.epoch == 0: \n",
    "            self.rec_start = len(rec.losses)\n",
    "        iters = range_of(rec.losses)\n",
    "        val_pos = rec.metric_names.index('valid_loss') - 1\n",
    "        val_losses = [v[val_pos] for v in rec.values]\n",
    "#         x_bounds = (0, (self.n_epoch - len(self.nb_batches)) * self.nb_batches[0] + len(rec.losses))\n",
    "        x_bounds = (0, len(rec.losses))\n",
    "        if self.epoch == 0:\n",
    "            y_min = min((min(rec.losses), min(val_losses)))\n",
    "            y_max = max((max(rec.losses), max(val_losses)))\n",
    "        else:\n",
    "            y_min = min((min(rec.losses[self.rec_start-1:]), min(val_losses)))\n",
    "            y_max = max((max(rec.losses[self.rec_start-1:]), max(val_losses)))\n",
    "        margin = (y_max - y_min) * .05\n",
    "        y_bounds = (y_min - margin, y_max + margin)\n",
    "        self.update_graph([(iters, rec.losses), (self.nb_batches, val_losses)], x_bounds, y_bounds)\n",
    "\n",
    "    def after_fit(self):\n",
    "        plt.close(self.graph_ax.figure)\n",
    "        if self.plot_metrics: self.learn.plot_metrics(final_losses=self.final_losses)\n",
    "\n",
    "    def update_graph(self, graphs, x_bounds=None, y_bounds=None, figsize=(6,4)):\n",
    "        if not hasattr(self, 'graph_fig'):\n",
    "            self.graph_fig, self.graph_ax = plt.subplots(1, figsize=figsize)\n",
    "            self.graph_out = display(self.graph_ax.figure, display_id=True)\n",
    "        self.graph_ax.clear()\n",
    "        if len(self.names) < len(graphs): self.names += [''] * (len(graphs) - len(self.names))\n",
    "        for g,n in zip(graphs,self.names): self.graph_ax.plot(*g, label=n)\n",
    "        self.graph_ax.legend(loc='upper right')\n",
    "        self.graph_ax.grid(color='gainsboro', linewidth=.5)\n",
    "        if x_bounds is not None: self.graph_ax.set_xlim(*x_bounds)\n",
    "        if y_bounds is not None: self.graph_ax.set_ylim(*y_bounds)\n",
    "        self.graph_ax.set_title(f'Losses\\nepoch: {self.epoch +1}/{self.n_epoch}')\n",
    "        self.graph_out.update(self.graph_ax.figure)        \n",
    "ShowGraphCallback2 = ShowGraph"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## SaveModel"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#export\n",
    "\n",
    "class SaveModel(TrackerCallback):\n",
    "    \"A `TrackerCallback` that saves the model's best during training and loads it at the end with a verbose option.\"\n",
    "    _only_train_loop,order = True,TrackerCallback.order+1\n",
    "    def __init__(self, monitor='valid_loss', comp=None, min_delta=0., fname='model', every_epoch=False, at_end=False,\n",
    "                 with_opt=False, reset_on_fit=True, verbose=False):\n",
    "        super().__init__(monitor=monitor, comp=comp, min_delta=min_delta, reset_on_fit=reset_on_fit)\n",
    "        assert not (every_epoch and at_end), \"every_epoch and at_end cannot both be set to True\"\n",
    "        # keep track of file path for loggers\n",
    "        self.last_saved_path = None\n",
    "        store_attr('fname,every_epoch,at_end,with_opt,verbose')\n",
    "\n",
    "    def _save(self, name): self.last_saved_path = self.learn.save(name, with_opt=self.with_opt)\n",
    "\n",
    "    def after_epoch(self):\n",
    "        \"Compare the value monitored to its best score and save if best.\"\n",
    "        if self.every_epoch:\n",
    "            if (self.epoch%self.every_epoch) == 0: self._save(f'{self.fname}_{self.epoch}')\n",
    "        else: #every improvement\n",
    "            super().after_epoch()\n",
    "            if self.new_best:\n",
    "                pv(f'Better model found at epoch {self.epoch} with {self.monitor} value: {self.best}.', self.verbose)\n",
    "                self._save(f'{self.fname}')\n",
    "\n",
    "    def after_fit(self, **kwargs):\n",
    "        \"Load the best model.\"\n",
    "        if self.at_end: self._save(f'{self.fname}')\n",
    "        elif not self.every_epoch: self.learn.load(f'{self.fname}', with_opt=self.with_opt)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Uncertainty-based data augmentation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#export\n",
    "class UBDAug(Callback):\n",
    "    r\"\"\"A callback to implement the uncertainty-based data augmentation.\"\"\"\n",
    "    \n",
    "    def __init__(self, batch_tfms:list, N:int=2, C:int=4, S:int=1): \n",
    "        r'''\n",
    "        Args:\n",
    "            batch_tfms:   list of available transforms applied to the combined batch. They will be applied in addition to the dl tfms.\n",
    "            N:            # composition steps (# transforms randomly applied to each sample)\n",
    "            C:            # augmented data per input data (# times N transforms are applied)\n",
    "            S:            # selected data points used for training (# augmented samples in the final batch from each original sample)\n",
    "        '''\n",
    "        \n",
    "        self.C, self.S = C, min(S, C)\n",
    "        self.batch_tfms = L(batch_tfms)\n",
    "        self.n_tfms = len(self.batch_tfms)\n",
    "        self.N = min(N, self.n_tfms)\n",
    "        \n",
    "    def before_fit(self):\n",
    "        assert hasattr(self.loss_func, 'reduction'), \"You need to pass a loss_function with a 'reduction' attribute\"\n",
    "        self.red = self.loss_func.reduction\n",
    "    \n",
    "    def before_batch(self):\n",
    "        if self.training:\n",
    "            with torch.no_grad():\n",
    "                setattr(self.loss_func, 'reduction', 'none')\n",
    "                for i in range(self.C):\n",
    "                    idxs = np.random.choice(self.n_tfms, self.N, False)\n",
    "                    x_tfm = compose_tfms(self.x, self.batch_tfms[idxs], split_idx=0)\n",
    "                    loss = self.loss_func(self.learn.model(x_tfm), self.y).reshape(-1,1)\n",
    "                    if i == 0:\n",
    "                        x2 = x_tfm.unsqueeze(1)\n",
    "                        max_loss = loss\n",
    "                    else: \n",
    "                        losses = torch.cat((max_loss, loss), dim=1)\n",
    "                        x2 = torch.cat((x2, x_tfm.unsqueeze(1)), dim=1)\n",
    "                        x2 = x2[np.arange(x2.shape[0]).reshape(-1,1), losses.argsort(1)[:, -self.S:]]\n",
    "                        max_loss = losses.max(1)[0].reshape(-1,1)\n",
    "                setattr(self.loss_func, 'reduction', self.red)\n",
    "            x2 = x2.reshape(-1, self.x.shape[-2], self.x.shape[-1])\n",
    "            if self.S > 1: self.learn.yb = (torch_tile(self.y, 2),)\n",
    "            self.learn.xb = (x2,)\n",
    "\n",
    "    def __repr__(self): return f'UBDAug({[get_tfm_name(t) for t in self.batch_tfms]})'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: left;\">\n",
       "      <th>epoch</th>\n",
       "      <th>train_loss</th>\n",
       "      <th>valid_loss</th>\n",
       "      <th>accuracy</th>\n",
       "      <th>time</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>1.852533</td>\n",
       "      <td>1.804390</td>\n",
       "      <td>0.166667</td>\n",
       "      <td>00:11</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from tsai.data.all import *\n",
    "from tsai.models.all import *\n",
    "dsid = 'NATOPS'\n",
    "X, y, splits = get_UCR_data(dsid, return_split=False)\n",
    "tfms = [None, Categorize()]\n",
    "dsets = TSDatasets(X, y, tfms=tfms, splits=splits)\n",
    "dls = TSDataLoaders.from_dsets(dsets.train, dsets.valid, batch_tfms=[TSStandardize()])\n",
    "model = create_model(InceptionTime, dls=dls)\n",
    "TS_tfms = [TSMagScale(.75, p=.5), TSMagWarp(.1, p=0.5),  TSWindowWarp(.25, p=.5), \n",
    "           TSSmooth(p=0.5), TSRandomResizedCrop(.1, p=.5), \n",
    "           TSRandomCropPad(.3, p=0.5), \n",
    "           TSMagAddNoise(.5, p=.5)]\n",
    "\n",
    "ubda_cb = UBDAug(TS_tfms, N=2, C=4, S=2)\n",
    "learn = Learner(dls, model, cbs=ubda_cb, metrics=accuracy)\n",
    "learn.fit_one_cycle(1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Weight per sample loss"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This process shows an example of how the weights could be calculated. This particular regression method was published in: \n",
    "\n",
    "Yang, Y., Zha, K., Chen, Y. C., Wang, H., & Katabi, D. (2021). Delving into Deep Imbalanced Regression. arXiv preprint arXiv:2102.09554.    \n",
    "(https://arxiv.org/pdf/2102.09554.pdf)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# export\n",
    "from scipy.ndimage import gaussian_filter1d\n",
    "from scipy.signal.windows import triang\n",
    "from scipy.ndimage import convolve1d\n",
    "\n",
    "\n",
    "def get_lds_kernel_window(lds_kernel=\"gaussian\", lds_ks=9, lds_sigma=1):\n",
    "    r\"\"\"Function to determine the label distribution smoothing kernel window\n",
    "\n",
    "    lds_kernel (str): LDS kernel type\n",
    "    lds_ks (int): LDS kernel size (should be an odd number).\n",
    "    lds_sigma (float): LDS gaussian/laplace kernel sigma\n",
    "    \"\"\"\n",
    "\n",
    "    assert lds_kernel in ['gaussian', 'triang', 'laplace']\n",
    "    half_ks = (lds_ks - 1) // 2\n",
    "\n",
    "    if lds_kernel == 'gaussian':\n",
    "        base_kernel = [0.] * half_ks + [1.] + [0.] * half_ks\n",
    "        kernel_window = gaussian_filter1d(\n",
    "            base_kernel, sigma=lds_sigma) / max(gaussian_filter1d(base_kernel, sigma=lds_sigma))\n",
    "    elif lds_kernel == 'triang':\n",
    "        kernel_window = triang(lds_ks)\n",
    "    else:\n",
    "        def laplace(x): return np.exp(-abs(x) / lds_sigma) / (2. * lds_sigma)\n",
    "        kernel_window = list(map(laplace, np.arange(-half_ks, half_ks + 1))) / \\\n",
    "            max(map(laplace, np.arange(-half_ks, half_ks + 1)))\n",
    "\n",
    "    return kernel_window\n",
    "\n",
    "\n",
    "def prepare_LDS_weights(labels, n_bins=None, label_range=None, reweight='inv', lds_kernel='gaussian', lds_ks=9, lds_sigma=1, \n",
    "                        max_rel_weight=None, show_plot=True):\n",
    "    \n",
    "    assert reweight in {'inv', 'sqrt_inv'}\n",
    "    labels_shape = labels.shape\n",
    "    if n_bins is None:\n",
    "        labels = labels.astype(int)\n",
    "        n_bins = np.max(labels) - np.min(labels)\n",
    "    num_per_label, bin_edges = np.histogram(labels, bins=n_bins, range=label_range)\n",
    "    new_labels = np.searchsorted(bin_edges, labels, side='left')\n",
    "    new_labels[new_labels == 0] = 1\n",
    "    if reweight == 'sqrt_inv':\n",
    "        num_per_label = np.sqrt(num_per_label)\n",
    "    lds_kernel_window = get_lds_kernel_window(lds_kernel=lds_kernel, lds_ks=lds_ks, lds_sigma=lds_sigma)\n",
    "    smoothed_value = convolve1d(num_per_label, weights=lds_kernel_window, mode='constant')\n",
    "    if show_plot:\n",
    "        plt.bar(bin_edges[:-1], num_per_label / num_per_label.sum(), width=(bin_edges[1]-bin_edges[0]), color='lime', edgecolor='black', label='original')\n",
    "        plt.plot(bin_edges[:-1], smoothed_value / smoothed_value.sum(), color='red', label='smoothed')\n",
    "        plt.title(f\"Label distribution by bin (reweight={reweight})\")\n",
    "        plt.legend(loc='best')\n",
    "        plt.show()\n",
    "    num_per_label = smoothed_value[new_labels.flatten() - 1].reshape(*labels_shape)\n",
    "    weights = 1 / num_per_label\n",
    "    weights[num_per_label == 0] = 0\n",
    "    if max_rel_weight is not None: \n",
    "        weights = np.clip(weights, None, np.min(weights) * max_rel_weight)\n",
    "    weights = weights / weights.sum() * len(labels)\n",
    "    return torch.Tensor(weights)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAEICAYAAABRSj9aAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAt1klEQVR4nO3deZwUxd3H8c8XBFYQEIVAwrWInAFFXMALTcQDL/BW1AhPVGIiiUceD0i8UOMRY0wUDx5jMInECzUkEg80KqgYFsQLFjkEWVDkULmR4/f8Ub0wLDO7s7uzO7u9vzeveTHdXV1dUzvzm5rq6mqZGc455+KrTrYL4JxzrnJ5oHfOuZjzQO+cczHngd4552LOA71zzsWcB3rnnIs5D/TViKTXJV1c2ftK+oGkwoTljyX9oDzHTZL3+ZJeTlg2SftnIu8ov3WS9stUfgn5LpJ0TAby2aVuk2x/SNL1Fcj/J5LuLe/+mSDp35KGppm23O/pUvLd5X1WgXwOkPR2JspUne2R7QLEkaRFwMVmNjnbZUmHmX2/tDSScoFPgXpmtrWEvB4HHs9EuSS9DvzNzB5JyH+vTOSdLWZ2aXn3lVQf+DVwSOZKVHZmdkIm8kn3PZWiDBl5n5nZB5K+lnSKmf2zovlVV96idxkjyRsOlWswUGBmS5Nt9Povt8eBn2S7EJXJA30VktRM0r8krZD0VfS8TbFkHSX9V9IaSf+QtE/C/odIejtqgbyfbneLpD0ljYuOORvoU2z7jm4LSX0l5UfHXy7pnijZm9H/X0fdJ4dKGibpLUm/l7QKuClaN7VYEU6UtFDSSkm/lVQnOtZNkv6WUI7cqKtnD0m3Af2B+6Pj3R+l2dEVJKmppL9E9blY0q8T8h4maaqku6PX/amk0lqifSTNjtL/WVJOlNdHkk5JKGe96LUcVEKdj4rSLJJ0fsL6cZJujZ7/QFKhpF9K+lLS55L+p4TynQC8kaS+LpL0GfBatP7HkuZEr+MlSe2j9TdLui/hNayX9NtoeU9Jm4rebyW91xK7YyTVlfS76LV+KmlE0d8wodzto/fJWkkvS2oerd/tPVXCay9ev7u8z6JjXippXlTmMQoaRMs9EtK2kLRR0neiVa8DAyQ1SPf4NY0H+qpVB/gz0B5oB2wE7i+W5kLgx8B3ga3AHwEktQZeAG4F9gH+F5ggqUUax70R6Bg9jgdK6l/9A/AHM2sSpX8qWn9k9P/eZraXmb0TLfcDFgItgdtS5HkakAf0JrRKf1xagc3sV8AUYER0vBFJkt0HNAX2A44i1F1ioOwHzAWaA3cBf5KkEg57PqF+OgKdCd0kAH8BLkhIdyLwuZm9lyKfVtExWxPqeqykLiWkbRqlvQgYI6lZirQ9o9dT3FFAN+B4SYOBUcDpQAtCHf49SvcG8IPoeR/gC3b+XQ8F5prZ6jK+1y4hfAH1Ivx9T02S5jzC3+U7QP0oP0jynpJ0RBSYUz2OSFE3ACdHr+sA4GzgeDPbDDwLDElIdzbwhpl9CRD9QtoCpPob1Xge6KuQma0yswlmtsHM1hIC41HFkv3VzD4ys/XA9cDZkuoSAs0kM5tkZtvN7BUgnxB0SnM2cJuZrTazJURfHilsAfaX1NzM1pnZtFLyXmZm95nZVjPbmCLNndGxPwPuZdcPXblEdXIuMNLM1prZIuB3wI8Ski02s/8zs23AY4Qvz5YlZHu/mS0xs9WEv01ROf9G+FXSJFr+EfDXUop4vZltNrM3CEHz7BTptgCjzWyLmU0C1pE64OwNrE2y/iYzWx/V/6XA7WY2J+r3/g3QK2rVvwN0krQvIcj+CWgtaS/C+7Do10JZ3mtnExoGhWb2FXBHkjR/NrNPovI9RfhSSMrMpprZ3iU8iv9aTHSHmX0dvc/+k3Cc8YT3SpHzonWJ1hLqN5Y80FchSQ0lPRx1M6wh/HTdOwpaRZYkPF8M1CO0DtsDZyW2boAjCMGrNN9Lkm8qFxFaswWSpks6uZS8l5SyvXiaxVF5Kqo5oW4SX8tiQsu4yBdFT8xsQ/S0pJO5SctpZsuAt4AzJO1NaMGWdCLwq+iLere8klhV7ETkhhLK+BXQuJRytwf+kPAeWQ0IaB0F2nxCUD+SENjfBg5n10Bflvda8fdWsvfDFwnPS3p9FZXqOP8BGkrqp3ACuBfwXLF9GwNfV1K5ss5P3lStXxJaa/3M7AtJvYD3CB/EIm0TnrcjtPhWEj5AfzWzS8px3M+jfD9OyDcpM5sHDFHo6z4deCZqAaaa5jSd6U+LH3tZ9Hw90DAhXasy5L2SUDftgdkJeSc9UZmm4nW/LGH5MeBiwmfmnVQnRCPNJDVKCPbtgI8qUK4iHxC+hItLrKclhF9vqb6I3gCOBg4CpkfLxwN92dlnXpb32udA4nmmtqkSllJuACT1B/5dwj4nmNmUMhwDM9sm6SnCL7TlwL+iX9RFx2xN6FJK1i0WC96irzz1JOUkPPYgtBo2Ek4+7UPoOy/uAkndJTUERgPPRF0PfwNOkXR8dAIsJzqZV/xkbjJPASMVTga3AX6eKqGkCyS1MLPt7GzhbAdWRP+XZwz71dGx2wKXA09G62cBR0pqJ6kpMLLYfstTHS+qk6eA2yQ1jromriLUU3ldJqlN9Lf5VUI5AZ4n9EFfTuizL83NkupHgetk4OkKlKvIJHbv6ivuIcLf+vuw44T1WQnb3yCcy5htZt8STkReDHxqZiuiNGV5rz0FXC6pdfRr59oyvJ7d3lNmNiXqr0/1KFOQTzAeOIdwHqZ4t81RwGtRf34seaCvPJMIQb3ocROhf3pPQmt0GvBikv3+Cowj/AzNAX4BEPWtF51oW0FodV1Nen/DmwndB58CL1Ny//JA4GNJ6wgnZs81s41R18dtwFvRz/myjOX+BzCDENhfIPQNE/X9Pkloqc4A/lVsvz8AZyqMHkl2XuHnhF8FC4GphA/wo2UoV3HjCfWzEFhAOBlJVNaNwASgA+HkXkm+IHSzLCN08VxqZgUVKFeRfwJdJaXs+jKz54A7gSei7sGPCF1NRd4mvAeLWu+zgU0Jy2V9r/0foc4+IPw6nUQYRLCttBdTwfdUmZjZu4T3yvfY/RfD+YQvyNiS33jEufRIugHobGYXlJq48sowHOhuZldkqwwlURjC+pCZtc92WdIh6QDgYTNLe2hnTeSB3rk0RN057wE/MrM3S0tfW0jaE/ghoVXfkvCrZ1p1/SKqrbzrxrlSSLqE0H3xbw/yuxGha/ArwhfhHOCGrJbI7cZb9M45F3PeonfOuZirduPomzdvbrm5udkuhnPO1SgzZsxYaWZJp0RJK9BLGkgY6lYXeMTM7ii2/VLgMsKQqnXAcDObHV2FNoedFyJMK22a1tzcXPLz89MplnPOuYiklFe8lxroo8vzxwDHAoXAdEkTzWx2QrLxZvZQlH4QcA9hPDbAAjPrVc6yO+ecq6B0+uj7AvPNbGF0Jd0ThIspdjCzNQmLjUjvsnjnnHNVIJ1A35pdJyoqZNeJowCQdJmkBYTpYH+RsKmDpPckvRFdDr4bScMV5kDPX7FiRbIkzjnnyiljJ2PNbAxhLu3zCPN4DyVMeNTOzFZJOhh4XtL3i/0CwMzGAmMB8vLy/NeAczGyZcsWCgsL2bRpU7aLEgs5OTm0adOGevXqpb1POoF+KbvOSNeGkmcIfAJ4ECCaJGhz9HxG1OLvTJgq1TlXCxQWFtK4cWNyc3Mp+b4vrjRmxqpVqygsLKRDhw5p75dO1810ws0KOijcnPhcYGJiAkmdEhZPAuZF61sUzbUuaT+gE2HCKOdcLbFp0yb23XdfD/IZIIl99923zL+OSm3Rm9lWSSOAlwjDKx81s48ljQbyzWwiMELhnqNbCJdCF92q7khgtKQthOlIL43u3uOcq0U8yGdOeeoyrT766BZnk4qtuyHh+eUp9ptAmOTIOedclvgUCM45lwGLFi1i/Pid9zQZN24cI0Yku6d9el5//XVOPrm0O3mmxwO9cxnQKrcVknZ7tMotfndEF1fFA3114oHeuQxYvnh5uEyw2GP54uXZLZhj/fr1nHTSSRx44IH06NGDJ598ktzcXEaOHEmvXr3Iy8tj5syZHH/88XTs2JGHHgo3mzIzrr76anr06EHPnj158sknS1x/3XXXMWXKFHr16sXvf/97AJYtW8bAgQPp1KkT11xzzY4yvfzyyxx66KH07t2bs846i3Xr1gHw4osv0rVrV3r37s2zz5Z2I7P0VbtJzZxzMXbFFTBrVmbz7NUL7r035eYXX3yR733ve7zwwgsAfPPNN1x77bW0a9eOWbNmceWVVzJs2DDeeustNm3aRI8ePbj00kt59tlnmTVrFu+//z4rV66kT58+HHnkkbz99ttJ199xxx3cfffd/Otf4Y6Y48aNY9asWbz33ns0aNCALl268POf/5w999yTW2+9lcmTJ9OoUSPuvPNO7rnnHq655houueQSXnvtNfbff3/OOeecjFWRt+idc7HWs2dPXnnlFa699lqmTJlC06ZNARg0aNCO7f369aNx48a0aNGCBg0a8PXXXzN16lSGDBlC3bp1admyJUcddRTTp09PuT6ZAQMG0LRpU3JycujevTuLFy9m2rRpzJ49m8MPP5xevXrx2GOPsXjxYgoKCujQoQOdOnVCEhdckLk7VnqL3jlXdUpoeVeWzp07M3PmTCZNmsSvf/1rBgwYAECDBg0AqFOnzo7nRctbt27NyLET861bty5bt27FzDj22GP5+9//vkvaWZn+pZPAW/TOuVhbtmwZDRs25IILLuDqq69m5syZae3Xv39/nnzySbZt28aKFSt488036du3b8r1jRs3Zu3ataXme8ghh/DWW28xf/58IJxD+OSTT+jatSuLFi1iwYIFALt9EVSEt+idc7H24YcfcvXVV1OnTh3q1avHgw8+yJlnnlnqfqeddhrvvPMOBx54IJK46667aNWqVcr1++67L3Xr1uXAAw9k2LBhNGvWLGm+LVq0YNy4cQwZMoTNmzcDcOutt9K5c2fGjh3LSSedRMOGDenfv39aXxzpqHb3jM3LyzO/8YiraSQln5xbYZRGbTZnzhy6deuW7WLESrI6lTTDzPKSpfeuG+ecizkP9M45F3Me6J1zLuY80DvnXMx5oHfOuZjzQO+cczHngd45V6VSzfRZ3kemZgg98cQT+frrr0tMc8MNNzB58uRy5Z/JaYfLyi+Ycs5VqR0zfWYqP1VshlAzw8yYNGlSqWlHjx5doWNli7fonXOxd88999CjRw969OjBvffey6JFi+jSpQsXXnghPXr0YMmSJeTm5rJy5UoAbrnlFrp06cIRRxzBkCFDuPvuuwEYNmwYzzzzDAC5ubnceOON9O7dm549e1JQUADAf//7Xw499FAOOuggDjvsMObOnZudF53AW/TOuVibMWMGf/7zn3n33XcxM/r168dRRx3FvHnzeOyxxzjkkEN2ST99+nQmTJjA+++/z5YtW+jduzcHH3xw0rybN2/OzJkzeeCBB7j77rt55JFH6Nq1K1OmTGGPPfZg8uTJjBo1igkTsntHVQ/0zrlYmzp1KqeddhqNGjUC4PTTT2fKlCm0b99+tyAP8NZbbzF48GBycnLIycnhlFNOSZn36aefDsDBBx+840Yh33zzDUOHDmXevHlIYsuWLZXwqsrGu26cc7VSUeCviKJpiIumIAa4/vrr+eEPf8hHH33EP//5TzZt2lTh41RUWoFe0kBJcyXNl3Rdku2XSvpQ0ixJUyV1T9g2MtpvrqTjM1l455wrTf/+/Xn++efZsGED69ev57nnnqN///4p0x9++OE7AvS6det23DEqXd988w2tW7cGwl2mqoNSA72kusAY4ASgOzAkMZBHxptZTzPrBdwF3BPt2x04F/g+MBB4IMrPOVdLtWzfEkTGHi3btyzxeL1792bYsGH07duXfv36cfHFF6ecQhigT58+DBo0iAMOOIATTjiBnj177rgrVTquueYaRo4cyUEHHZSxG5hUVKnTFEs6FLjJzI6PlkcCmNntKdIPAS40sxOKp5X0UpTXO6mO59MUu5rIpylOrSZOU7xu3Tr22msvNmzYwJFHHsnYsWPp3bt3tou1Q1mnKU7nZGxrYEnCciHQr3giSZcBVwH1gaMT9p1WbN/WaRzTOeeyZvjw4cyePZtNmzYxdOjQahXkyyNjo27MbAwwRtJ5wK+BoenuK2k4MBygXbt2mSqSc86Vy/jx47NdhIxK52TsUqBtwnKbaF0qTwCnlmVfMxtrZnlmlteiRYs0iuScq0lqe/dVJpWnLtMJ9NOBTpI6SKpPOLk6MTGBpE4JiycB86LnE4FzJTWQ1AHoBPy3zKV0ztVYOTk5rFq1yoN9BpgZq1atIicnp0z7ldp1Y2ZbJY0AXgLqAo+a2ceSRgP5ZjYRGCHpGGAL8BVRt02U7ilgNrAVuMzMtpWphM65Gq1NmzYUFhayYsWKbBclFnJycmjTpk2Z9vGbgzuXAT7qxmWb3xzcOedqMQ/0zjkXcx7onXMu5jzQO+dczHmgd865mPNA75xzMeeB3jnnYs4DvXPOxZwHeuecizkP9M45F3Me6J1zLuY80DvnXMx5oHfOuZjzQO+cczHngd4552LOA71zzsWcB3rnnIs5D/TOORdzHuidcy7mPNA751zMeaB3zrmY80DvnHMxl1aglzRQ0lxJ8yVdl2T7VZJmS/pA0quS2ids2yZpVvSYmMnCO+ecK90epSWQVBcYAxwLFALTJU00s9kJyd4D8sxsg6SfAncB50TbNppZr8wW2znnXLrSadH3Beab2UIz+xZ4AhicmMDM/mNmG6LFaUCbzBbTOedceaUT6FsDSxKWC6N1qVwE/DthOUdSvqRpkk5NtoOk4VGa/BUrVqRRJOecc+kqteumLCRdAOQBRyWsbm9mSyXtB7wm6UMzW5C4n5mNBcYC5OXlWSbL5JxztV06LfqlQNuE5TbRul1IOgb4FTDIzDYXrTezpdH/C4HXgYMqUF7nnHNllE6gnw50ktRBUn3gXGCX0TOSDgIeJgT5LxPWN5PUIHreHDgcSDyJ65xzrpKV2nVjZlsljQBeAuoCj5rZx5JGA/lmNhH4LbAX8LQkgM/MbBDQDXhY0nbCl8odxUbrOOecq2Qyq15d4nl5eZafn5/tYjhXJpIg2UdJUN0+Yy6eJM0ws7xk2/zKWOecizkP9M45F3Me6J1zLuY80DvnXMx5oHfOuZjzQO+cczHngd4552LOA71zzsWcB3rnnIs5D/TOORdzHuidcy7mPNA751zMeaB3zrmY80DvnHMx54HeOedizgO9c87FnAd655yLOQ/0zjkXcx7onXMu5jzQO+dczHmgd865mEsr0EsaKGmupPmSrkuy/SpJsyV9IOlVSe0Ttg2VNC96DM1k4Z1zzpWu1EAvqS4wBjgB6A4MkdS9WLL3gDwzOwB4Brgr2ncf4EagH9AXuFFSs8wV3znnXGnSadH3Beab2UIz+xZ4AhicmMDM/mNmG6LFaUCb6PnxwCtmttrMvgJeAQZmpujOOefSkU6gbw0sSVgujNalchHw73Lu65xzLsP2yGRmki4A8oCjyrjfcGA4QLt27TJZJOecq/XSadEvBdomLLeJ1u1C0jHAr4BBZra5LPua2VgzyzOzvBYtWqRbduecc2lIJ9BPBzpJ6iCpPnAuMDExgaSDgIcJQf7LhE0vAcdJahadhD0uWuecc66KlNp1Y2ZbJY0gBOi6wKNm9rGk0UC+mU0EfgvsBTwtCeAzMxtkZqsl3UL4sgAYbWarK+WVOOecS0pmlu0y7CIvL8/y8/OzXQznykQSJPsoCarbZ8zFk6QZZpaXbJtfGeucczHngd4552LOA71zzsWcB3rnnIs5D/TOORdzHuidcy7mPNA751zMeaB3zrmY80DvnHMx54HeOedizgO9c87FnAd655yLOQ/0zjkXcx7onXMu5jzQO+dczHmgd865mPNA75xzMeeB3jnnYs4DvXPOxZwHeuecizkP9M45F3Me6J1zLubSCvSSBkqaK2m+pOuSbD9S0kxJWyWdWWzbNkmzosfETBXcOedcevYoLYGkusAY4FigEJguaaKZzU5I9hkwDPjfJFlsNLNeFS+qc8658ig10AN9gflmthBA0hPAYGBHoDezRdG27ZVQRueccxWQTtdNa2BJwnJhtC5dOZLyJU2TdGqyBJKGR2nyV6xYUYasnXPOlaYqTsa2N7M84DzgXkkdiycws7FmlmdmeS1atKiCIjnnXO2RTqBfCrRNWG4TrUuLmS2N/l8IvA4cVIbyOeecq6B0Av10oJOkDpLqA+cCaY2ekdRMUoPoeXPgcBL69p1zzlW+UgO9mW0FRgAvAXOAp8zsY0mjJQ0CkNRHUiFwFvCwpI+j3bsB+ZLeB/4D3FFstI5zzrlKJjPLdhl2kZeXZ/n5+dkuhnNlIgmSfZQE1e0z5uJJ0ozofOhu/MpY55yLOQ/0zjkXcx7onXMu5jzQO+dczHmgd865mPNA75xzMeeB3jnnYs4DvXPOxZwHeuecizkP9M45F3Me6J1zLuY80DvnXMx5oHfOuZjzQO+cczHngd4552LOA71zzsWcB3rnnIs5D/TOORdzHuidcy7mPNA751zMeaB3zrmY80DvnHMxl1aglzRQ0lxJ8yVdl2T7kZJmStoq6cxi24ZKmhc9hmaq4M4559JTaqCXVBcYA5wAdAeGSOpeLNlnwDBgfLF99wFuBPoBfYEbJTWreLGdc86lK50WfV9gvpktNLNvgSeAwYkJzGyRmX0AbC+27/HAK2a22sy+Al4BBmag3M4559KUTqBvDSxJWC6M1qUjrX0lDZeULyl/xYoVaWbtnKuNWuW2QtJuj1a5rbJdtGqrWpyMNbOxZpZnZnktWrTIdnGcc9XY8sXLwdjtsXzx8uwWrBpLJ9AvBdomLLeJ1qWjIvs655zLgHQC/XSgk6QOkuoD5wIT08z/JeA4Sc2ik7DHReucc85VkVIDvZltBUYQAvQc4Ckz+1jSaEmDACT1kVQInAU8LOnjaN/VwC2EL4vpwOhonXOx0gK4ZCz86yS4+5ewx5Zsl8i5nWRm2S7DLvLy8iw/Pz/bxXCudIWF8Oyz8OyzbHvjDeoCn7WFdkvgnyfD2U/BpoZQ3T5jNZ2k0C+/24baXdeSZphZXrJt1eJkrHM1yuLFMHAgtG0Ll18OK1dyK3DA+9B+Mfz0ATjpBXj5OGia7bI6hwd659JnBo8/DgccAG+/DaNHQ0EBfPQRNwEfHgAIHvopnPsE9HsX3gD44ousFts5D/TOpeOrr2DIELjgAujZE95/H66/Hrp0SZr86bNDq74jwOGHw4IFVVpc5xJ5oHeuNK+9FlrxEybAbbfBG29Ahw6l7jb5WDga4OuvQ7CfNauSC+pcch7onSvJ6NEwYAA0bAjvvAOjRkHdumnvPh1g6lSoVw/69IERI2C5X9jjqpYHeudS+fbbEOgHDYKZMyEv6YCG0nXrBvn5cMkl8NBD0LEj3HQTrF2b0eI6l4oHeudSmT8ftm2Dc86BRo0qllfLlvDAAzB7Npx4Itx8M+y/P4wZE75QnKtEHuidS6WgIPzftWvm8uzcGZ56Ct59F7p3D105J5yQufydS8IDvXOpzJkT/u/cOfN59+0bTvJeeSX85z+wcWPmj+FcxAO9c6kUFISLovbaq3Lyl6BfvzA+f968yjmGc3igdy61goJwIrUyFY3Dnzu3co9TGzTA56lPwQO9c8mYhUCfyf75ZIq6hYrOB7jy24zPU5+CB3rnklm2DNatq/xA37AhtGvnLXpXqTzQO5dMZYy4SaVrV2/RZ1Cz1XDiCySf4bKW8kDvXDJFI26qItB36RJa9LV4it1MqEO4J8AnneGFk+GBn4G2Z7tU1cMe2S6Ac9VSQQE0aQKtquBEXteuoZto2TJo3bryjxdDfd+F+4E+P4E3joSPvw8/exDqfwvDs124asADvXPJFI24kSr/WIkjb2IQ6Fvltkp6ArRl+5Z8sSizUza3+BJuHwkXPRpuRj1kPDxxbtj25XfgppuhAcDWrbBH7Q133nXjXDJVMeKmSNFxYtJPv3zx8ioZ/XLcS6Gb5sK/wJ3XQFfgiSGAwuPmm2DUbXABwHnnwZbae39HD/TOFbd2LSxdWnWB/nvfCxdl+cibtHVYGFrun7WDnh/CdXfCuiTpbh8FvwR4+mk46yzYvLmKS1o9eKB3rriqHHEDoXuoS5fYtOgzpVVuq6QXQDUAnjkzpDn1eZhbyp/pngYwAuAf/2BSTg71a+HFVB7onSuuqgM97Bx543ZI1QV0H9D7PfjRX+HT/dLIaDOMMbj0QTgRuHgMte5iqrQCvaSBkuZKmi/puiTbG0h6Mtr+rqTcaH2upI2SZkWPhzJcfucyr6AgnLjr2LHqjtm1a7jp+IYNVXfMGmjYn+ES4LZRYQhlWTz8E5h6OIz6DdSvZT04pQZ6SXWBMcAJQHdgiKTuxZJdBHxlZvsDvwfuTNi2wMx6RY9LM1Ru5ypPQUEI8vXqVd0xi0be+ORmKR04K4yNfxW4YXQ5MlC430ubpXDRnzJbtuounRZ9X2C+mS00s2+BJ4DBxdIMBh6Lnj8DDJCqYlyac5WgKiYzKy5mI28yrenXMOEMWL0PDAG2p383x128OqB2turTCfStgSUJy4XRuqRpzGwr8A2wb7Stg6T3JL0hqX+yA0gaLilfUv6KFSvK9AKcy6itW0Oruir75wE6dQonZb2ffjfaDo8NhXafwVlPQ4UiRC1t1Vf2ydjPgXZmdhBwFTBeUpPiicxsrJnlmVleixYtKrlIzpVg4cIw3rqqA/2ee0L79t6iL6bvu/D0WTB4Ivzv3fDOYRXP89UBMOWIqFVf8exqhHQC/VKgbcJym2hd0jSS9gCaAqvMbLOZrQIwsxnAAqASbtfjXIZkY8RNER95E3z7LYwfzzTg3UPgmMlw8w3wx19kKP/EVn2Gsqzu0gn004FOkjpIqg+cC0wslmYiMDR6fibwmpmZpBbRyVwk7Qd0AhZmpujOVYKiQF90crQq1fbJzb76Cm65BXJz4fzz2RsYcR+0KQxTGZDBs36vHR216qFWXERVaqCP+txHAC8Bc4CnzOxjSaMlDYqS/QnYV9J8QhdN0RDMI4EPJM0inKS91MxWZ/g1OJc5BQVhIrO99676Y3ftCuvXh6tya5uvv4b+/eGGG+DAA2HSJLoBY0bAusaVcLyiVj3An+LfWS+rZq2HvLw8y8/Pz3YxXG112GGQkxNu3F0GkpLPfy5I+zP22mswYAC88gocc0yZjp8NqSYvA8pWF5s3w8CB8NZb8MILcOyxIXkJdZqR9QZv1oH+rVvDggXQoEHy11JDSJphZnnJtvmVsc4VMQvz0Gejfx52HreG9NOnunK1TLZvh//5H3j9dRg3bkeQrxKCmyD8gop5q94DvXNFvvwydCFkK9B/97thcrPaNPJm1Cj4+9/h9tvDDJNV7DWAI46A3/wm1n31HuidK5Iw4ibVhFqVOgmWFL5kakiLvsIeeADuvBN++lO49trslePmm0Or/vbbs1eGSuaB3rkiCYG+quZU301tmcXyH/+An/8cBg2C++6rmhu8pHL00XDBBXDrrRDT84Me6J0rUlAADRtCmzbZK0PXrrBkSRh9E1dvvw1DhkCfPqHbpm455zPIpPvuC6OtfvQj2Lgx26XJOA/0rlZL7KJ58d57mblhA8pm4Ckav//JJ9krQyWpD2FM4w9+EG628s9/hi/WbGoQRveoWTOOXboUCgq4p2HD2M1T74He1WqJXTRd28Oc8yj7yJFMqmEjb9LV912YAaE//JxzYNo0qA7TnWxmx99/ssH9l4ULgbrGbJ56D/TOAXtugNzFUFBFA25SnezNPemY0F9dU/rpDRquhzrbkm9uuB7uuRLeORSaQBgn/9e/QvPmVVnKtF17J8zbH8YBrFmT5dJkTu29LbpzCTpHPSVVFeh3/JIoZrG+hA4dql+L/ttv4YorYNasEADXrOEroPEeUHc7bK0Ln38XlrYO09sWXgFffgcu+T/osCi0lEeOgbUnnpjNV1GqDY3CzcanHgZceWVsxtd7oHcO6Bo1oKsq0JeoOo68ufJKePBB+OEPw3j/Jk14bNw41oyCtY2h8dowJ02bwnB3ouP/BI3XwdzO0P9NmNqfcPuiGmDaoeHOSaMefRROPRVOOSXbRaowD/TOAd3mwHbBvE7ZLgmhn/7NN8NVo3WqQe/qI4+EMe/XXBPGvUeuGDcObkmSXsBaaLwG1u0FVg1eQlndBIw68EC4+GL46KPqcT6hAmrgn8C5zOtaAJ92gM05Gc64aFRHsUeJunQJ944tLMxwYcrh7bfhZz+D444LV4+WwdomNTPIA2wB+Mtf4Jtv4KyzQtdVDVZD/wzOldNXX8GKFbBp0y7TAXctgDmVcffAhFEdac8HU11G3ixbBmecAe3aVZ/x7lXpgANCH/0bb4Srd6vZBJBl4YHe1Q5btsCNN4bRHt/5TrijU4MGrAAWdoAeH1WT/nnYOZY+m/30mzfD6afD2rXw/POwzz7ZK0s2nX8+XH89PPoo/O532S5NuXkfvYu/Tz4Jl7hPnx4+uIccEgLYmjU8eccdNO4PDTfA4+dnu6CRVq2gSZPstejNQnfNu+/ChAnQo0d2ylFd3HRT+NK95hro3DlM21DTmFm1ehx88MHmXEZs32728MNmDRuaNWtm9uSTuyUBLOm/EtYnU558Ssy/Tx+zAQMqs3ZSu/9+MzC7/voSk2Wq7jKZf5nWl6Ws69eb5eWZNWpk9t57ZazQqgHkW4q46l03Lp6+/BIGD4af/AQOO4xeDeuic84p20nRbOraNYz2WLWq6o5pBvfeC5dfDiefHFqypL64K9aKn0Rv1Ijv5uezdNPG0KL//PNsl7BMPNC7eDGDv/0NevaEl1+G3/8eXnqJ95eurPhNMqrSCSfA8uVhgrVLLoEPP6zc423aFG4AcuWVIciPH79jaGdGbjBS0yQ5if6Fwcnbtocv38GDw8ioGsIDvYuPWbPgyCPDDITt24c++SuuqB5j0ctqyJAQ3C+8EB5/PIwAOfroML3vthTzDZTXsmVw1FHw2GPhhPWzz0LjyrhRa803C8LfIz8fDj443PaxJkjVp5Oth/fRuzJbvdrsssvM6tQxa97c7JFHzLZt2yUJme5DL6ZS81+50uyOO8zatg195/vtF849bNpU8bp7+22zVq1C3/Ozz1b9aytBJo+b8l95X8OkSWYdO4a/xxlnmC1aVNG/RIVRQh991gN78UdFA33L9i2T/ci0Og3rlGl9y/YtK1QOV3mK/sb1wS4G+xJsK9gfwTq3aZF0nxod6Its2WL29NPhRC2YtWlj9sc/mm3YUPZKXLnS7MEHzerXD18cH36Y8rPjgT7Fa9i40ezWW8323DM8brklrMuSkgK9wvbqIy8vz/IrcJcXSeRsgLuuCfNvNFkTPV6FJl3C87qJv3y/BL4Tni5vGS6amdMNZt8MT33wQRhOVcPvDh8rGzZwRqNGnDEETv4XNFkLU46AEffDBwcCCo2X4iQl71cWZV6fjfx3YRa6DG65BaZOhZYt4Ze/DBf1NGoUbpyxZs2OIaSsXh2Gas6eHR5z5oST1RBuxv3EE7DPPhl9DeV+bYnJK7s85cxrt9fw2Weh/p95BvbbL3QXHntsuB6iCk9aS5phZnlJt6VT8ZIGAn8A6gKPmNkdxbY3AP4CHAysAs4xs0XRtpHARcA24Bdm9lJJx8pEoK+3OQTtNU0SHm/DmrPDBExb6iXs8BBwKcig9dIw58l+C6FOUbXUqQMdO0K3btC9e3h06xZGRey1V7nL6UqwbRusW7djlkTWroUFC+C552DSJNi4kZX7wvOnwjNnwkvHEz6YUPMDfQ7hRGAxdRrWYfuG7butP7VlM57reTBMngz164e6S9WHv/fe0K0bj3/4HjPXbeIj4FXCB3MHD/Qlr0/x92nZviVfPPI3uOqqnSfO27YNAf/YY2HAgEqfL6dCgV5SXeAT4FjCDKTTgSFmNjshzc+AA8zsUknnAqeZ2TmSugN/B/oC3wMmA53NLOXZpEwE+oq+CXI2Quem0H0LdCPMxtcN6AwkfkfQvv3OL4DE/5s1K3f5a43Nm2HevJ0tzKL/P/00BPkkPgeeBSYAb26Bbcku90vxQQRqRqAvZ+DpB5wFbALWAOzTmDv/+GC48KppU+jUKVyIVTQ0MhtljUOgT+MLoAMhWB4LHFNH7L092qFduzCKqnVrHnrxBeat3UAhMA94L8qmZfuWfLHoixSFLVlFA/2hwE1mdny0PBLAzG5PSPNSlOYdSXsAXwAtgOsS0yamS3W86hDoU63fYwt0rA8FEybsDExz5oSr5hLvM9moUXZvdlwTbNgQZmeEUFf77Re+JPffP7Q8mzTh4quuYs2T4RfZl9+BWb0Ik2RVxQe3pgT6cgRVD/SVkFeK9XUE26ZNC7+4PvkkTFRXWMj6Tz6hUZTm1aPhmFd35lPe7vSSAn06UyC0BpYkLBcSGhBJ05jZVknfAPtG66cV27d1kgIOB4ZHi+skVeza71QxtvT1zYGVqdJvBeYCOuOMko8f5xs7F6+jTDALXTMLFuy+7ZwU+5T0PVr+v39a61NeLJTm+6iq1pd4UVP2y5Te+6iyy1nJx9gO6JBDSjg48Nqu+5a5jnZqn2pDtZjrxszGAmOzXQ5J+am+EV3gdVQ6r6PSeR2VLpN1lM6VJEuBtgnLbaJ1SdNEXTdNCSdl09nXOedcJUon0E8HOknqIKk+cC4wsViaicDQ6PmZwGvRuM6JwLmSGkjqAHQC/puZojvnnEtHqV03UZ/7COAlwvDKR83sY0mjCQP0JwJ/Av4qaT6wmvBlQJTuKWA2oYv7spJG3FQDWe8+qgG8jkrndVQ6r6PSZayOqt0FU8455zKrBs725Jxzriw80DvnXMx5oAck/VZSgaQPJD0nae+EbSMlzZc0V9LxWSxmVkk6S9LHkrZLyiu2zesoImlgVA/zJV2X7fJUB5IelfSlpI8S1u0j6RVJ86L/a+3l5JLaSvqPpNnRZ+zyaH3G6sgDffAK0MPMDiBM9zASIJrC4Vzg+8BA4IFoSoja6CPgdODNxJVeRztFr3sMcAJh5owhUf3UduMI741E1wGvmlknwpQ7tflLcSvwSzPrDhwCXBa9bzJWRx7oATN72cy2RovTCOP9AQYDT5jZZjP7FJhPmLen1jGzOWaW7Iplr6Od+gLzzWyhmX0LPEGon1rNzN4kjMZLNBh4LHr+GHBqVZapOjGzz81sZvR8LTCHMINAxurIA/3ufgz8O3qebPqH3aZwqOW8jnbyukhfSzMruvHqF0DLbBamupCUCxwEvEsG66haTIFQFSRNBlol2fQrM/tHlOZXhJ9Rj1dl2aqLdOrIuUwzM5NU68d5S9qLMDnrFWa2JnGuoorWUa0J9GZ2TEnbJQ0DTgYG2M6LC2rVFA6l1VEKtaqOSuF1kb7lkr5rZp9L+i7hFkC1lqR6hCD/uJk9G63OWB151w0U3VjlGmCQmSXe2t2ncCid19FO6UwX4oLEaVOGArX2F6NC0/1PwBwzuydhU8bqyK+MBaKpGxoQJmIDmGZml0bbfkXot99K+En17+S5xJuk04D7CPcZ+BqYlXCPAq+jiKQTgXvZOV3IbdktUfZJ+jvwA8K0u8uBG4HngaeAdsBi4GwzK37CtlaQdAQwBfiQMLMxwChCP31G6sgDvXPOxZx33TjnXMx5oHfOuZjzQO+cczHngd4552LOA71zzsWcB3rnnIs5D/TOORdz/w/ZeuCzUYTPfQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAEICAYAAABRSj9aAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAA1VUlEQVR4nO3dd5gUVdbA4d9hSBIEBQQlDSqC44wiDKArQcWAEVEQUFZRkfXbRVfdlWAWcVdcV3TVXWVFwQCCYEBFRXSVYCJIThIFCRKVIGHgfH/cGmianunume6pnprz8vRDd9XtqtNVNadv37p1S1QVY4wxwVXK7wCMMcYklyV6Y4wJOEv0xhgTcJbojTEm4CzRG2NMwFmiN8aYgLNEn2Ai8oWI9Ez2e0XkXBFZE/J6voicW5D1Rlj29SIyIeS1isjJiVi2t7wdInJiopYXstyVInJBApZz2LaNMP8FEXmgEMv/g4g8XdD3J4KIfCQiN8ZYtsDHdCoq7P4LWc7tIjIoETElW2m/A0hVIrIS6KmqE/2OJRaqelq0MiKSDqwAyqhqTj7LegN4IxFxicgXwOuq+lLI8islYtl+UdXbCvpeESkL3A+clbiI4qeqlyRiObEeU8kmIj1wf6+topUtzP4L819gqYj8U1V/TtAyk8Jq9OYwImJf/snVAVikqj9FmmnbP35+bTNV3Q18BNzgx/rjYYk+TiJyjIh8ICIbRWSr97xOWLGTROQ7EflVRN4TkWND3n+WiHwlIttEZHaszS0icpSIDPPWuQBoHjb/YLOFiLQQkene+jeIyFNesUne/9u85pOzRaSHiEwVkcEishl42Js2JSyES0VkuYhsEpF/iEgpb10Pi8jrIXGke009pUXkMaA18Jy3vue8MgebgkSkioi86m3PVSJyf8iye4jIFBF50vvcK0QkWk20uYgs8Mq/IiLlvWXNE5ErQuIs432WM/PZ5vd6ZVaKyPUh04eJyEDv+bkiskZE/iIiP4vIOhG5KZ/4LgG+jLC9bhGRH4HPvek3i8hC73N8IiL1vemPiMizIZ9hp4j8w3t9lIjszj3e8jvWQptjRCRNRP7pfdYVItI7dx+GxF3fO062i8gEEanuTT/imMrns4dv35NF5EsR+cVb96iQeReKyCJv3nNeudx4w4/ZUcALwNleDNuirDem/SciLUVkvYikhby3o4jMCVncF8BlsX5mv1iij18p4BWgPlAP+A14LqzMDcDNwPFADvAvABGpDXwIDASOBf4KjBWRGjGs9yHgJO9xMZBf++ozwDOqerRXfrQ3vY33f1VVraSqX3uvWwLLgZrAY3kssyOQDTTF1Upvjhawqt4HTAZ6e+vrHaHYs0AV4ESgLW7bhSbKlsBioDrwBDBURCSf1V6P2z4nAafgmkkAXgW6h5S7FFinqt/nsZxa3jpr47b1EBFplE/ZKl7ZW4DnReSYPMpmeZ8nXFvgVOBiEekA3AtcDdTAbcORXrkvgXO9582B9Rzar2cDi1V1S5zH2q24L6AmuP17VYQy1+H2y3FAWW95EOGYEpFW3pdLXo/c5pVHgQnAMUAd3LGA9yXyNm7fVQeWAeeExRN6zHYHbgO+9mKoGiH+/ETcf6r6LbATOD9sO4wIeb0QOCPO9RU5S/RxUtXNqjpWVXep6nZcYmwbVuw1VZ2nqjuBB4BrvVpBd2C8qo5X1QOq+ikwHZd0orkWeExVt6jqarwvjzzsA04WkeqqukNVv4my7LWq+qyq5qjqb3mUGeSt+0fgaaBbDDHny9smXYH+qrpdVVcC/wR+H1Jslar+V1X3A8NxX54181nsc6q6WlW34PZNbpyv436VHO29/j3wWpQQH1DVPar6JS5pXptHuX3AAFXdp6rjgR1AXl8KVYHtEaY/rKo7ve1/G/B3VV3otXv/DWji1eq/BhqKSDVckh0K1BaRSrjjMPfXQjzH2rW4isEaVd0KPB6hzCuqusSLbzTuSyEiVZ2iqlXzeeT+WtyHqzCdoKq7Q6ZfCsxX1TGqug93vK0PW00sx2ys8tt/I/GOIRGp7MU2MuS923FfEinNEn2cRKSCiLworpnhV9xP16qhP++A1SHPVwFlcDWT+kDn0NoN0AqXvKI5IcJy83ILrja7SESmicjlUZa9Osr88DKrvHgKqzpu24R+llW4mlWug3/gqrrLe5rfydyIcarqWmAqcI2IVMXVYPM74bzV+6I+YlkRbA47Ebkrnxi3ApWjxF0feCbkGNkCCFDbS2rTcUm9DS6xf4Wr8YYm+niOtfBjK9LxEJpo8/t88eiD+1zfies1lvsr8bB41I28GB5TLMdsrPLbfyOAq0WkHO4X1kxVDT1eKwO/JDCWpLATP/H7C+7bvqWqrheRJsD3uAM2V92Q5/VwNYZNuIPzNVW9tQDrXectd37IciNS1R+AbuLauq8Gxng1wLyGKo1lCNPwda/1nu8EKoSUqxXHsjdxqFa3IGTZEU9Uxih8268NeT0c6Ik77r/O64So5xgRqRiS7OsB8woRV645uC/hcKHbaTXu11teX0Rf4poTzgSmea8vBlpwqM08nmNtHa7pJFfdvApGiRsAEWmNO0mZl0tUdbKqrsc1G+E150wUkUkcOtZzlycRYgpfb1KG4VXVBSKyClcxCG+2AdfcNjsZ604kq9Hnr4yIlA95lMZ9g/+GO/l0LK7tPFx3EckQkQrAAGCM1/TwOnCFiFzsnQAr750MCj+ZG8looL+4k8F1gNvzKigi3UWkhqoeALZ5kw8AG73/C9KH/R5v3XWBP+NOgAHMAtqISD0RqQL0D3vfhrzW522T0cBjIlLZa5q4G7edCupPIlLH2zf3hcQJ8C6uDfrPuDb7aB4RkbJe4roceKsQceUaz5FNfeFewO3r0+DgCevOIfO/xJ3LWKCqe3EnBHsCK1R1o1cmnmNtNPBnEant/drpG8fnOeKY8pJ4pXwek73P1Tkknq24ZH0A10x2mohc7f3N3cGRFYhwG4A64rqvJtoI3DHThiOPgbbk/6WWEizR5288LqnnPh7GtRcehauNfgN8HOF9rwHDcD93y+MOVLy29dwTbRtxta57iG0/PIJrPliBO4GVX/tye2C+iOzAnZjtqqq/eU0fjwFTvZ/z8fTlfg+YgUvsH+LahvHafkfhaqozgA/C3vcM0Elc75FI5xVux/0qWA5Mwf1RvRxHXOFG4LbPctxJvIG5M7xmj7FAA9zJvvysxyWftbgmnttUdVEh4sr1PtBYRPJs+lLVd4BBwJte8+A8XI0y11e4YzC39r4A2B3yOt5j7b+4bTYH9+t0PK4Twf5oH6aQx1Rz4FvvOB0H/FlVl6vqJqAz7lzBZqAhrtktP5/jfnGuF5FNccQQi5G4hP65FxsA4np0XYr7pZjSRO3GI6YEEZEHgVNUtXvUwsmLoReQoap3+hVDfsR1YX1BVev7HUsuiXDhnd9E5Hagrqr28TuWaKyN3pQYXnPOLRzeq6fIqeoQP9cfTkSOAs7D1epr4poj3/E1qGJAVZ/1O4ZYWdONKRFE5FZc88VHqjopWvkSRnBNg1txTTcLgQd9jaiQvF48OyI8ro/+7uCJqelGRNrj2lrTgJdU9fGw+W1wbden49qDx4TNPxrXjvhuHhfNGGOMSZKoNXqvf/jzuJNBGbhuexlhxX4EenBk16NcjxJyosgYY0zRiaWNvgWwVFWXA4jIm7iz+bn9nvGuaEREDoS/WUSa4dr9PsZdQp+v6tWra3p6egxhGWOMyTVjxoxNqhpxOJVYEn1tDr8KbQ1unImovAt2/om7HDumccLT09OZPn16LEWNMcZ4vAu7Ikr2ydg/4sbbyPMmDuC6m4kbbXH6xo0b8ytqjDEmTrHU6H/i8MuP6xD7JepnA61F5I+4sSPKisgOVe0XWsjrbjYEIDs72zr2G2NMAsWS6KfhRstrgEvwXXFjPkSlqqFjePcAssOTvDHGmOSKmuhVNUdEegOf4LpXvqyq80VkADBdVceJSHPcBRbH4MbXeERjuLWdMSb49u3bx5o1a9i9e7ffoQRC+fLlqVOnDmXKlIn5PSk3BEJ2drbayVhjgmPFihVUrlyZatWqIfneM8ZEo6ps3ryZ7du306BBg8PmicgMVY3Ys9GujDXGJNXu3bstySeIiFCtWrW4fx1ZojfGJJ0l+cQpyLa0RG+MMQFnid4YYxJg5cqVjBhxaBSYYcOG0bt3wYf2+uKLL7j88mh3AY2NJXpjEqBWei1E5IhHrfRoN0YyQRGe6FOJJXpjEmDDqg3uRnhhjw2rNvgbmGHnzp1cdtllnHHGGWRmZjJq1CjS09Pp378/TZo0ITs7m5kzZ3LxxRdz0kkn8cILLwCuh8s999xDZmYmWVlZjBo1Kt/p/fr1Y/LkyTRp0oTBgwcDsHbtWtq3b0/Dhg3p0+fQ/UkmTJjA2WefTdOmTencuTM7duwA4OOPP6Zx48Y0bdqUt9+OdhO02NmNR4wxRefOO2HWrMQus0kTePrpPGd//PHHnHDCCXz44YcA/PLLL/Tt25d69eoxa9Ys7rrrLnr06MHUqVPZvXs3mZmZ3Hbbbbz99tvMmjWL2bNns2nTJpo3b06bNm346quvIk5//PHHefLJJ/ngA3c3zWHDhjFr1iy+//57ypUrR6NGjbj99ts56qijGDhwIBMnTqRixYoMGjSIp556ij59+nDrrbfy+eefc/LJJ9OlS5eEbSKr0RtjAi0rK4tPP/2Uvn37MnnyZKpUqQLAlVdeeXB+y5YtqVy5MjVq1KBcuXJs27aNKVOm0K1bN9LS0qhZsyZt27Zl2rRpeU6PpF27dlSpUoXy5cuTkZHBqlWr+Oabb1iwYAHnnHMOTZo0Yfjw4axatYpFixbRoEEDGjZsiIjQvXvi7nZpNXpjTNHJp+adLKeccgozZ85k/Pjx3H///bRr1w6AcuXKAVCqVKmDz3Nf5+TkJGTdoctNS0sjJycHVeXCCy9k5MiRh5WdlehfOiGsRm+MCbS1a9dSoUIFunfvzj333MPMmTNjel/r1q0ZNWoU+/fvZ+PGjUyaNIkWLVrkOb1y5cps37496nLPOusspk6dytKlSwF3DmHJkiU0btyYlStXsmzZMoAjvggKw2r0xphAmzt3Lvfccw+lSpWiTJky/Oc//6FTp05R39exY0e+/vprzjjjDESEJ554glq1auU5vVq1aqSlpXHGGWfQo0cPjjnmmIjLrVGjBsOGDaNbt27s2bMHgIEDB3LKKacwZMgQLrvsMipUqEDr1q1j+uKIhY11Y0wCiIjraXPEDNdLoyRbuHAhp556qt9hBEqkbWpj3RhjTAlmid4YYwLOEr0xxgScJXpjjAk4S/TGGBNwluiNMSbgLNEbY4pUXiN9FvSRqBFCL730UrZt25ZvmQcffJCJEycWaPmJHHY4XnbBlDGmSB0c6TNRy5PCjRCqqqgq48ePj1p2wIABhVqXX6xGb4wJvKeeeorMzEwyMzN5+umnWblyJY0aNeKGG24gMzOT1atXk56ezqZNmwB49NFHadSoEa1ataJbt248+eSTAPTo0YMxY8YAkJ6ezkMPPUTTpk3Jyspi0aJFAHz33XecffbZnHnmmfzud79j8eLF/nzoEFajN8YE2owZM3jllVf49ttvUVVatmxJ27Zt+eGHHxg+fDhnnXXWYeWnTZvG2LFjmT17Nvv27aNp06Y0a9Ys4rKrV6/OzJkz+fe//82TTz7JSy+9ROPGjZk8eTKlS5dm4sSJ3HvvvYwdO7YoPmqeYqrRi0h7EVksIktFpF+E+W1EZKaI5IhIp5DpTUTkaxGZLyJzRCRxAywbY0wMpkyZQseOHalYsSKVKlXi6quvZvLkydSvX/+IJA8wdepUOnToQPny5alcuTJXXHFFnsu++uqrAWjWrBkrV64E3Hj3nTt3JjMzk7vuuov58+cn5XPFI2qiF5E04HngEiAD6CYiGWHFfgR6AOH30doF3KCqpwHtgadFpGohYzbGmEKrWLFioZeROwxx7hDEAA888ADnnXce8+bN4/3332f37t2FXk9hxVKjbwEsVdXlqroXeBPoEFpAVVeq6hzgQNj0Jar6g/d8LfAzUCMhkRtjTAxat27Nu+++y65du9i5cyfvvPMOrVu3zrP8OeecczBB79ix4+Ado2L1yy+/ULt2bcDdZSoVxJLoawOrQ16v8abFRURaAGWBZfG+1xgTHDXr1wQhYY+a9Wvmu76mTZvSo0cPWrRoQcuWLenZs2eeQwgDNG/enCuvvJLTTz+dSy65hKysrIN3pYpFnz596N+/P2eeeWbCbmBSWFGHKfba3Nurak/v9e+BlqraO0LZYcAHqjombPrxwBfAjar6TYT39QJ6AdSrV6/ZqlWrCvRhjPGLDVOct+I4TPGOHTuoVKkSu3btok2bNgwZMoSmTZv6HdZB8Q5THEuvm5+AuiGv63jTYiIiRwMfAvdFSvIAqjoEGAJuPPpYl22MMcnQq1cvFixYwO7du7nxxhtTKskXRCyJfhrQUEQa4BJ8V+C6WBYuImWBd4BXw2v5xhiTqkaMCO9XUrxFbaNX1RygN/AJsBAYrarzRWSAiFwJICLNRWQN0Bl4UURy+xNdC7QBeojILO/RJBkfxBiTukp681UiFWRbxnTBlKqOB8aHTXsw5Pk0XJNO+PteB16POypjTGCUL1+ezZs3U61aNXcuwxSYqrJ582bKly8f1/vsylhjTFLVqVOHNWvWsHHjRr9DCYTy5ctTp84R9ep8WaI3xiRVmTJlaNCggd9hlGg2qJkxxgScJXpjjAk4S/TGGBNwluiNMSbgLNEbY0zAWaI3xpiAs0RvjDEBZ4neGGMCzhK9McYEnCV6Y4wJOEv0xhgTcJbojTEm4CzRG2NMwFmiN8aYgLNEb4wxAWeJ3hhjAs4SvTHGBJwlemOMCThL9MYYE3CW6I0xJuBiSvQi0l5EFovIUhHpF2F+GxGZKSI5ItIpbN6NIvKD97gxUYEbY4yJTdRELyJpwPPAJUAG0E1EMsKK/Qj0AEaEvfdY4CGgJdACeEhEjil82MYYY2IVS42+BbBUVZer6l7gTaBDaAFVXamqc4ADYe+9GPhUVbeo6lbgU6B9AuI2xhgTo1gSfW1gdcjrNd60WMT0XhHpJSLTRWT6xo0bY1y0McaYWKTEyVhVHaKq2aqaXaNGDb/DMcaYQIkl0f8E1A15XcebFovCvNcYY0wCxJLopwENRaSBiJQFugLjYlz+J8BFInKMdxL2Im+aMcaYIhI10atqDtAbl6AXAqNVdb6IDBCRKwFEpLmIrAE6Ay+KyHzvvVuAR3FfFtOAAd40Y4wxRURU1e8YDpOdna3Tp0/3Owxj4iIiEOlPSSDV/sZMMInIDFXNjjQvJU7GGmOMSR5L9MYYE3CW6I0xJuAs0RtjTMBZojfGmICzRG+MMQFnid4YYwLOEr0xxgScJXpjjAk4S/TGGBNwluiNMSbgLNEbY0zAWaI3xpiAs0RvjDEBZ4neGGMCzhK9McYEnCV6Y4wJOEv0xhgTcJbojTEm4CzRG2NMwFmiN8aYgLNEb4wxARdToheR9iKyWESWiki/CPPLicgob/63IpLuTS8jIsNFZK6ILBSR/gmO3xhjTBRRE72IpAHPA5cAGUA3EckIK3YLsFVVTwYGA4O86Z2BcqqaBTQD/pD7JWCMMaZoxFKjbwEsVdXlqroXeBPoEFamAzDcez4GaCciAihQUURKA0cBe4FfExK5McaYmMSS6GsDq0Ner/GmRSyjqjnAL0A1XNLfCawDfgSeVNUt4SsQkV4iMl1Epm/cuDHuD2GMMSZvyT4Z2wLYD5wANAD+IiInhhdS1SGqmq2q2TVq1EhySMYYU7LEkuh/AuqGvK7jTYtYxmumqQJsBq4DPlbVfar6MzAVyC5s0MYYY2IXS6KfBjQUkQYiUhboCowLKzMOuNF73gn4XFUV11xzPoCIVATOAhYlInBjjDGxiZrovTb33sAnwEJgtKrOF5EBInKlV2woUE1ElgJ3A7ldMJ8HKonIfNwXxiuqOifRH8IYY0zexFW8U0d2drZOnz7d7zCMiYuIuD5mR8yAVPsbM8EkIjNUNWLTuF0Za4wxAWeJ3hhjAs4SvTHGBJwlemOMCThL9MYYE3CW6I0xxUqt9FqIyBGPWum1/A4tZZX2OwBjjInHhlUbInZl3SAbij6YYsJq9MYYE3CW6I0xJuAs0RtjTMBZojfGmICzRG+MMQFnid4YYwLOEr0xxgScJXpjjAk4S/TGGBNwluiNMSbgLNEbY0zAWaI3xpiAs0RvjDEBZ4neGGMCzhK9McYEXEyJXkTai8hiEVkqIv0izC8nIqO8+d+KSHrIvNNF5GsRmS8ic0WkfALjN8YYpxx2Q5I8RE30IpIGPA9cAmQA3UQkI6zYLcBWVT0ZGAwM8t5bGngduE1VTwPOBfYlLHpjjMm1B3dDkrDHhlV2Q5JYavQtgKWqulxV9wJvAh3CynQAhnvPxwDtRESAi4A5qjobQFU3q+r+xIRujDEmFrEk+trA6pDXa7xpEcuoag7wC1ANOAVQEflERGaKSJ9IKxCRXiIyXUSmb9y4Md7PYIwxJh/JPhlbGmgFXO/931FE2oUXUtUhqpqtqtk1atRIckjGmCBrOgPeuQpq/Ox3JKkjlkT/E1A35HUdb1rEMl67fBVgM672P0lVN6nqLmA80LSwQRtjTEQK/7oDrnoPnr3d72BSRyyJfhrQUEQaiEhZoCswLqzMOOBG73kn4HNVVeATIEtEKnhfAG2BBYkJ3RiTimql1/Kt90v7j+Gcr+C75tBlNFw9NumrLBZKRyugqjki0huXtNOAl1V1vogMAKar6jhgKPCaiCwFtuC+DFDVrSLyFO7LQoHxqvphkj6LMSYFbFi1wf21h0+X5Pd+GXg/LG8A534BU1rBv/8IXyZ9rakvaqIHUNXxuGaX0GkPhjzfDXTO472v47pYGmNM0nQEms2EG4fBbxWgxzCY0Qye8TmuVGBXxhpjir1S+2EAsKgRvHG9mzb3dFfDvx5gXHhrc8liid4Yk5LyauuPpMsoyAQeegT2h7RT/L0/zBZY26EDx5Tgq2Yt0RtjUtLBtv7wR5i0HHjkIZgNvBXWgLyvLNykcFwaDL7xyGWVlKtmLdEbY4q1G16FhkvhAUAjZLTvgcf7QY/hcMn4I+eXBOJ6QaaO7OxsnT59ut9hGBMXEYlY20Qg1f7Gki1R2yK/5eROL7sHlpwCG2pCy2nkWb7sbndituo2yFgA248uWEypTERmqGp2pHlWozemsLZt42pgyK2wsj6M7kzkhGMS7pahUP9HuH9g/uX2loP/+w/U+Qk6jSma2FKJJXpjCmLhQhg4EFq1gurVGQt0fgtW14XOY+DW//odYPAdtcsl+Emt4dMLo5ef0gqWnehO3JY0luiNiceOHXD33ZCZCQ88AHv2QP/+nANU3wRtJsGEC+Gpu+HEZX4HG2w3vQInrPNq85E74xxOYFQXaPcZVC9hYydaojcmVu+9BxkZMHgw3HorrF8P06bBo4/yFa5bn5aCm1+GnNIw/Eb7A0umrm/CnCyY3Cb294zqAqX3wzUlbGgEOw6NiWb1aujYEa66CqpUgalT4YUXoGbNiMV/qgN/eh5aTYW/Fm2kJcbxa+GcqUd2p4xmzunuoqqS1nxjid6Y/Lz8sqvFf/IJPP44zJwJv/td1LeNuA7e6gSPAsyZk/QwS5qr34ZSCmM6xflGr/mm7ZdQa11SQktJluiNycvevXDbbXD66TB/PvTtC2XKxPZecb08tgD8/veuLd8kTOe3YN5psOjU+N87qov7kihJvW8s0RuTlyVLYN8++NOfoEGDuN++uTr0BFejf+ihIwusW+fa/adMKXSoJUlNoPXkAtTmPQszYG5myWq+iWn0SmNKpLlz3f+ZmQVexIcAPXvCE0/ASSfB5s3uBO5338GaNa5Q1aqwZQvkMY6LOdzVuBp5vO3zod7sCo/d7+6iVBJYjd6YvMybB6VLQ+PGhVvOU09Bejr06gX9+7safps2rvfOnXfCtm3wU/hN21JfPIOOJVInYGFjWJBR8GWM6uL+L8R3RbFiNXpj8jJ3LpxyCpQtW7jlVK4Mn3/umoKaNYNq1Q7N+/JLePppWLAA6hSv+mVeNxiJqU97AdX42d2m7rHOhVvPspNhRlPoOjNRkaU2q9Ebk5d58yArKzHLSk+Hiy46PMmD69EDLtGbqDq+425zV9D2+VCjukALgOXLC7+wFGeJ3phIduyAFSsK1T4fkxo1oHp116vHRNVpDCwG5ibg+3f0tblPRhd+YSnOEr0xkeQm3mQneoDTTrMafQyqbYLz/gdjICHNQ6vS4RuAUcHvfmOJ3pgI7rr0IgBO6tgx+XckyshwiT4gw+Umy1XvuuEL3krgMkcBzJrlzp8EmCV6YyKov+VXdlaAFftJ/h2JMjJcz5t1JehSzQLo/BYsPcndSSpRDn5pBLxWb4nemAgygfmnRb5jUcLZCdmojt3sRp0sTN/5SH4CaN3aEj2AiLQXkcUislRE+kWYX05ERnnzvxWR9LD59URkh4jYGE+mWMgC5hVB8zzg2ujBEn0+Orznmm0S0dvmCF26uHMy8+YlYeGpIWqiF5E04HngEiAD6CYi4Zcq3AJsVdWTgcHAoLD5TwEfFT5cYxIr0kU/x4lQk8T07IjJccfBscdaz5t8dBoDK9JhZtMEL7gcHNe7NznAoKys5J6L8VEsNfoWwFJVXa6qe4E3gQ5hZToAw73nY4B24l0iJyJXASsAO4pNyjl40U/I47TP3bwiq9GLHDoha45QdStcMNFrtkn0xVh7YKPC+Cug+wlQKofknYvxUSyJvjawOuT1Gm9axDKqmgP8AlQTkUpAX+CR/FYgIr1EZLqITN+4sYTd+sWknCxviJsiS/Tgmm/mz7eeNxFcNwLK7ktSs41n+I1Qey1c+Gny1uGnZJ9qehgYrKo78iukqkNUNVtVs2vUqJHkkIzJX+Y82ASsT+Kv9/AmoztefBG2biWz3nHJW2kxUxp47F549nb4rjlMa568dX1wOWw+Fm4cHr1scRRLov8JqBvyuo43LWIZESkNVAE2Ay2BJ0RkJXAncK+I9C5cyMYkV9ZcmAdJHbMlvMlogVeTPG7NpuSttDhZuZJJwL1/h6G3uAulkrk/9paDkd3cEAtVtiVvPX6JJdFPAxqKSAMRKQt0BcaFlRkH3Og97wR8rk5rVU1X1XTgaeBvqvpcYkI3JgnU1ejnFvFq53sdbwoxIGNwjBkDTZqQAXR5E3r9F3ZVTP5qh/WA8nvg2gCOiBA10Xtt7r2BT4CFwGhVnS8iA0TkSq/YUFyb/FLgbuCILpjGFAf1foTKO7wafRFaXwu2Vi3hif6339wdvTp3hkaNOBMY3aXoVj+jmbtrVY9hRbfOohLTMMWqOh4YHzbtwZDnu4kytLOqPlyA+IwpUgdPxBb1isWNr37aV0W94hSxbx9cfrkbzrlPHxg4kBWFHR46XuJOyv6jD5xStGtOOrsy1pgQmV6G9+PSmfmnleAa/V//6pL8K6/AoEGx35s3wV7vDvtLwQ2+rD15LNEbEyJrLvxYF371Yd0LMqAGQEnrYvzKK/Cvf8Fdd0GPHr6Gsv54+ORiL9Hv3+9rLIlkid6YEJnzivCK2DAHb42XqlfIjhjh7oiVSN9849rlL7jA3Vc3BQzr4XUh/N//fI4kcSzRG+MpvQ9OXZjgC6XKEfN9VXN73qTkFbIrVsD118O558LFF8OMGYVf5tq1cPXV7haKb77p7s+bAsZdCVsBhg3zOZLEsURvjKfhD+4KzITW6PdwxBALEe+zCqw9wV1SnpKJPvcuTPfd55J8djajgUaLCri83btdkv/1V3jvvSNvseijPeXdOC+8/baLLwAs0RvjOXgitiiHPgglsABSs+lm9Gho0QIGDnT3WH3wQdrjfoW8dAvUimcofVX4v/+Db7+FV18tmrt4xWkYuO6ebyXyNif+sURvjCdrLuSkwaLG/sUwH1KvRr90KcycCdd6N1k9+mh45BFOBP51B3R/Haa0guPXxri8p55yzSIPPeRq9SnoO4BGjWB4MMZEsERvjCdzHvzQ0P1098sCgJ9/hk0pNBRCbrNN58MvldkE3D0YWk+G436GiRdA9Wgdhv72N9eV8ppr4MEHoxT2WY8eMHmy+6Ir5izRG+PJmutjs43nYF0+lWr1o0bB2WdDvXoRZ09rAZd9COkrYcJFUDVSIYUnwLXxX389jBwJpVI8/XTv7oaQfv11vyMptBTf0sYUjQo74cTl/nWtzHWwdT5VEv2iRTBnjrsLUz4mt3EDgp02311CX2n7oXml9sN/b4V7AP70J9cu79MFUXGpU8f1Mho5stgPH22J3pQcP/0ETZq45oMDBw6bdepCKKX+1+jXAFSqlDqJfvRoV6vtFH0w+AkXQ5dR0BzXRbH8b1B2D7zZFXoOhUcBnn029Wvyobp1gyVL4Pvv/Y6kUIrRFjemkPr1g9mzXfNB+/auLdzjy81G8pJKd5saPRpatYLa4fcaiuzdjm4Y27ZfwthrXMLvPAbu/ic8CO5Lozi55hr362PECL8jKRRL9KZk+Ppr19Z6770wZIg7yXbGGZzrzc6cB7+Vh2Un+RmkJ/duU36bP989cnvbxGgE8IcX4dKP3C0Abx4Kg+9OTohJd+yxcMkl7oKusF+BxUlqXIpmTDIdOMCsc9tQA2j0t7+xE8gCRq9fz0RgwCNwxmw3BMGBNH9DBVyN/pVXYMsWl2j84jXbZA16hHm33x7XW1+6FbYcC78eDRMvTFJ8yeRd0QzQBXcBVdu0NBbXr8n6let9Da0grEZvgm/4cJrszaHva7DTuzJ1rkL2dngDeORhuOAz/0/EHpThDXqzcKF/Mai6RN+2LfPWbIr56t5Qb19TTJM8HHZF8/s7YEdFuK5X8b1puCV6E2y//gr9+/MV8Mb1h8/aWcm1J9/0svtD/vx8PwKMIDfR+9l8M3eu63ETpbdNSbCrIrx7FXR+C4pBX6GILNGbYBs4EDZs4M+Q5z1Hh90EVX6B135flIHlo149qFjR3xOyo0e73jEpeuVqURvZDY7dChf5HUgBWaI3wbVkCTz9NNx0E9OjFD2QRlJvPh2XUqXg1FP9S/Sq7iKp88+H447zJ4YUM+Ei2FQNrvM7kAKyRG+C6y9/gfLlXb/54iYjA+bN8+dCnVmz3GX/cfa2CbKcMvBWZ+gAsHOn3+HEzRK9CaaPP4YPPoAHHoBatfyOJn6tW8O6dfDhh0W/7lGjIC3Nmm3CjOwGFQHGjfM7lLiJptilvdnZ2Tp9erQf2sbkY98+yMpyt4KbNw/KlXNd5SId6kLc0yP9zSR8+fv2ueF7RdyJ0aIaMkAVTjoJTjnFfVmShM8Wo0SuN89eQnG8Rw7AqjSoe/nl8P77eYXtGxGZoarZkeZZjd4Ez7BhsHgx/POfUK6c39EUTJky8OST7nO8+GLRrfedd9zdpLp3L7p1FhNayrshyccfw+bNfocTl5gSvYi0F5HFIrJURPpFmF9OREZ5878VkXRv+oUiMkNE5nr/p0oHNhNU+/a5NvnmzeGKK/yOpnAuv9ydEH34Ydi6Nfnry8mB/v3dieCuXZO/vmJoBLjtNHas36HEJWqiF5E04HngEiAD6CYiGWHFbgG2qurJwGBgkDd9E3CFqmbhuiy/lqjAjYnotddg5Uo31nlxG1clnIi7SceWLfDYY8lf38svu55Kf/97yty/NdXMAmjcuNiNfRNLjb4FsFRVl6vqXtyvlw5hZToAubdiGQO0ExFR1e9VNfe+M/OBo0SkmP6WNikvJ8clxKZN4bLL/I4mMc44A26+Gf71L1i2LHnr2bnT/XI45xy48srkrScIunWDSZNgzRq/I4lZLIm+NrA65PUab1rEMqqag7vHcfjdfq8BZqrqnvAViEgvEZkuItM3box2i5r81UqvhYgc8UirmBbX9FrpxbCnRgmR1z6+oUyZg/czLfa1+VCPPgply0LfvslbxzPPwLp1nDN1KlKq1GHb1YTp1s2dtB4yxO9IYlYkJ2NF5DRcc84fIs1X1SGqmq2q2TVq1CjUujas2hBxXI4Duw7ENb24jmlREkTax2n74P6G3k/roNVIjz/eDbE8dqyrSSbapk0waBDvAl8VYEybEqdhQzc0xKBB7mR5MRBLov8JqBvyuo43LWIZESkNVAE2e6/rAO8AN6hqEn97mpKsyyg45QcYAMGqzee6+253x6O77078cLmPPQY7dnBvYpcabE8/DRUqQK9exWL44lgS/TSgoYg0EJGyQFcg/IqBcbiTrQCdgM9VVUWkKvAh0E9VpyYoZmMOU2o/PPAozMmCd/0OJlkqVHAnSWfMgDfeSNxyV6yA55+Hm2/Gx7Eyi59atVz310mTYOhQv6OJTlWjPoBLgSXAMuA+b9oA4ErveXngLWAp8B1wojf9fmAn7hd17uO4/NbVrFkzLQxAI/4rwHSTmsL3WZeRqIJ2Gp33fkv2cVEkx93+/arZ2aq1a6tu2ZKITanavbtq+fKqa9Yk9DMk4m8q6fEU9jMcOKDatq1qlSqqa9cmZn8UAjBd88irgbsy1q+r+EzRCd3HcuDQOPJZc0HTiujK1SJe/kHffedu7de2LXz0UeG6Qc6a5Xoo9e0Lf/978q9ELeZXxuZOP+wzLFkCp5/uzguNHp3HSoqGXRlrAuuasXDaAnj0AXflYrHn3dkoz95hLVty0759MHEiz5QpU6DeYbm9lj4680w2q1L18cetd02swvdPo0bct2cPvPVWSg6LkKtEXRVxzBa47zE4fh3srAi7Krh2pV0D3ev3r4BlJ/sdpYmVHHBt8wsbu5EFAyH3zkZhDsiBg9OHAVl3w92DYV5Beoet2sBb18AlY91Nu3/JvZ+r5froIuyff+yFruUg649/hHPPhcqV/Ygsf3m16fj1SEob/QG0M+j649C9pdElJ6NrTkC3VkH3uB6xqqA/V0cbLyhYe6IpOrn7uMfLbr9d93r0duCIx0Vh22iTtPxYpqftQz+6GN0Lql9+GduGO3BA9dVXdTPob+XQvn9HS+UkP9Zo2y7afk5aPAlcVktQFVG9/fa4PmMikU8bve+JPfyR6ERfezX63hUuIXzbHM2afeSOK70XPXU+uq4muro2Wn9F/AelKTqANliG/loJ/V/bI5NVXu9J1B96spcf6/QqW9GFoFq9uuqKFflvtB9/VL30UlXQKaCNFhZtrPltu/z2c1LjSfRn693bJfuvv47rcyZKiUz0sh+97d/oL5XRnUehdxGWECLsuMw56OZj0B9OQmtZok9ZaaBTz3a/yOquii2ZBDHRo+jJoFq1qmpWluqvvx4eVE6O+wJ4/nnVypVVK1RQfeYZLeVTrIFP9L/+qlq3rmrDhqrbt8f1WROhxCX6muvQSa1cLX7CBa72F+uOa/k1ur0iOgdUN28uVCwmOe73mtq6jog9mQQ10QOqEyaopqWpnn++6h13uJp7o0aqZcpobrOknn++6rJl/scah2KX6FVVv/jC1epvuSWuz5oIJS7Rl92NftkavWEYyoH4d9z5E9HdoNqypS/fzCYf33yj+0Bfuz6+ZBLoRK+q+txzLtlXqqTapInqNdeo9umjOmSI6qRJrn0+VWKNUbFM9Kqq997rUutbb8X1eQurxCV6lEMJvoA77qrS6D7QT0HL4ZUDrVm/ZqHiM7GrWb/mwe0OaEXQJaArce3TEf+V47D3HPZI9B90CN+SZ8jnLR/jcWqJPgnLCtkPpUG/Ad0Cembt6jEd24nIL/kl+iD0PI6skF3F3s2Bm4fDBcDIq9ygWagNdlaUwgcvG9wTThL4PfBL1TzetIcIfz5FEa1PQj7v7pDPa8dpEQvZDzkK1/8ApSvBUz9tcre0DJPX4IvJ2m/BTfQJ8NoNcPu/oOO78N9bXb9t448O78KtL8GgvjDZ72CMiWLZydD7OTgX3CiXPrNEH8Vzt8NDD8NNw+DJv/odTcmUvgJe6gkzmsJDj/gdjTGxefUGGAnu/gjffutrLJboYzDgQXjmDnclog3lWrRuHgqzz4Cye6H767CvrN8RGRMjgf8DN7z0ddfB9u2+hWKJPhYCdw2GV38PjwH8+99+RxR8a9fyATC0J8xoBqfPgUWn+h2UMfH5BeD11919jDt1gm3bfInDEn2MtBTcMtQbiL93bxg50u+QgknVjbeemcl5wB3PQLvPYFW634EZU0CtWsGLL8Lnn0Pz5jBvXpGHYIk+DjlloAtAmzZwww3w4Yd+hxQsmze7Wk/37tC4MU2AZ+8gGKNSmpKtZ0/44gvYsQPOOouiHoPP/oTitBtg3Dg3BvU118CECX6HFAwbN8J558EHH7heCpMn84PfMRmTSOec4+4QdvrpjAYG9YG0nKJZtSX6gjj6aJfgGzd2NxywZF84GzdCu3bwww8wfjz06QNpaX5HZUzinXACfPEF/wH6/AM+bg/VNiV/tZboC6paNfjsM5fsO3SwZF9QmzYdSvIffOCeGxNkZcvyR1yPslZTYPylJP2ivhJ145GEy0327dq5ZD9uHFx4od9RFR+bNsH551uSNyXSKze722CW303Sb/piNfrCqlYNJk6ERo1cM86nn/odUfEQWpN//31L8qZEmt4cprRO/nos0SdC9eqHJ/tPPvE7otT2888usS9Z4pL8BRf4HZExgWaJPlFCk3379q6L4MqVfkeVWnbvhiefdNvIkrwxRSamRC8i7UVksYgsFZF+EeaXE5FR3vxvRSQ9ZF5/b/piEbk4gbGnnurVYdIkuPdeGDvWJbS//hW2bPE7Mn8dOACvvea2xz33wO9+B9OmWZI3pqjkNX5x7gNIA5YBJwJlgdlARliZPwIveM+7AqO85xle+XJAA285afmtLyk3B0/kuNOxjqm9erXqzTe7u81Urar6xBOqv/1WqM9WLE2Y4G6EAarNmql+9lnMb4173yR4P8ez/4tivYmI0+9Y45H0eHz8zInaRmHLzHM8+lh63bQAlqrqcgAReRPoACwIKdMBeNh7PgZ4TkTEm/6mqu4BVojIUm95X8ew3uKtTh0YOhTuvBP69XN9wx94AMqU8TuyoqMKO3dCejqMGAFdukApay00pqjFkuhrA6tDXq8BWuZVRlVzROQXoJo3/Zuw99YOX4GI9AJ6eS93iMjimKLPS15dlaJPrw5sijD98OJSwL5Qe/a4R/F2+DaKxcqVbvS+666Lf23x7suCvCfO6Xnu/ziPI9/iTI2YYjuOkh1nEawjhuMlr/Lx/q3Vz2tGSvSjV9UhwBC/4xCR6aqa7Xccqcy2UXS2jaKzbRRdIrdRLL+jfwLqhryu402LWEZESgNVgM0xvtcYY0wSxZLopwENRaSBiJTFnWwdF1ZmHHCj97wT8Ll3cmAc0NXrldMAaAh8l5jQjTHGxCJq043X5t4b+ATXA+dlVZ0vIgNwZ3nHAUOB17yTrVtwXwZ45UbjTtzmAH9S1SPvlJs6fG8+KgZsG0Vn2yg620bRJWwbiat4G2OMCSrr62aMMQFnid4YYwLOEj0gIv8QkUUiMkdE3hGRqiHzSs4QDvkQkc4iMl9EDohIdtg820aeaMOFlEQi8rKI/Cwi80KmHSsin4rID97/x/gZo59EpK6I/E9EFnh/Y3/2pidsG1midz4FMlX1dGAJ0B9ARDJwJ5ZPA9oD/xaRknrro3nA1cCk0Im2jQ7xPvfzwCW44T+6edunpBuGOzZC9QM+U9WGwGfe65IqB/iLqmYAZwF/8o6bhG0jS/SAqk5Q1dy7N36D6+8PIUM4qOoKIHcIhxJHVReqaqQrlm0bHXJwuBBV3QvkDhdSoqnqJFxvvFAdgOHe8+HAVUUZUypR1XWqOtN7vh1YiBtBIGHbyBL9kW4GPvKeRxr+4YghHEo420aH2LaIXU1VXec9Xw/U9DOYVOGN/Hsm8C0J3EYpMQRCURCRiUCtCLPuU9X3vDL34X5GvVGUsaWKWLaRMYmmqioiJb6ft4hUAsYCd6rqr6Fj5BR2G5WYRK+q+Q5+LiI9gMuBdnro4oISNYRDtG2UhxK1jaKwbRG7DSJyvKquE5HjgZ/9DshPIlIGl+TfUNW3vckJ20bWdIPrKQH0Aa5U1V0hs2wIh+hsGx0Sy3AhxgkdNuVGoMT+YvSGdB8KLFTVp0JmJWwb2ZWxgDd0QzncQGwA36jqbd68+3Dt9jm4n1QfRV5KsIlIR+BZoAawDZilqhd782wbeUTkUuBpDg0X8pi/EflPREYC5+KG3d0APAS8C4wG6gGrgGtVtUTeik1EWgGTgbnAAW/yvbh2+oRsI0v0xhgTcNZ0Y4wxAWeJ3hhjAs4SvTHGBJwlemOMCThL9MYYE3CW6I0xJuAs0RtjTMD9P1So59ZWloTzAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAEICAYAAABRSj9aAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAA0PUlEQVR4nO3deZgU1fXw8e+ZYRlQRIUJRBAGI6gIijAsRhYVF9wgoCi4QVyIP4NxSURwD0pURMAgUXGDuAEBMai4EV4FXJABEWUVEQSRYRFQdgbO+8etgabpbWa6p3pqzoenH7qrblWdrq45XX3r1r2iqhhjjAmuDL8DMMYYk1qW6I0xJuAs0RtjTMBZojfGmICzRG+MMQFnid4YYwLOEn2SichHInJDqpcVkTNFZHXI6wUicmZxthth3VeJyAchr1VEjk/Gur31bRWR45K1vpD1rhCRc5KwnoP2bYT5z4jIfSVY/59EZHhxl08GEXlXRHolWLbYx3Q6KunnF7KeW0TksWTElGoV/A4gXYnICuAGVZ3qdyyJUNWT45URkRzge6CiqhbEWNerwKvJiEtEPgJeUdXnQ9Z/eDLW7RdVvam4y4pIJeBeoE3yIio6Vb0gGetJ9JhKNRHpjft7bRuvbEk+vzDPActE5AlVXZekdaaEndGbg4iIffmnVhdgsar+GGmm7f+i82ufqepO4F3gWj+2XxSW6ItIRI4SkbdFZL2IbPKe1w0r9jsR+UJEfhGR/4rI0SHLtxGRT0Vks4h8lWh1i4hUEZHR3jYXAi3D5u+vthCRViKS520/X0SGesWme/9v9qpPTheR3iLyiYgME5GNwIPetJlhIVwoIstFZIOIPC4iGd62HhSRV0LiyPGqeiqIyCCgHfCUt72nvDL7q4JEpLqI/NvbnytF5N6QdfcWkZkiMsR739+LSLwz0ZYistAr/5KIZHnr+kZELgmJs6L3Xk6Lsc/v9sqsEJGrQqaPFpGHvednishqEfmriKwTkZ9E5I8x4rsA+DjC/rpeRH4ApnnTrxORRd77eF9E6nvT/y4iI0LewzYRedx7XUVEdhYeb7GOtdDqGBHJFJEnvPf6vYj0LfwMQ+Ku7x0nv4rIByJS05t+yDEV472H79/jReRjEdnibXtcyLxzRWSxN+8pr1xhvOHH7DjgGeB0L4bNcbab0OcnIq1FZK2IZIYs21VE5oes7iPgokTfs18s0RddBvASUB+oB+wAngorcy1wHfBboAD4J4CI1AHeAR4Gjgb+BkwUkewEtvsA8DvvcT4Qq371SeBJVT3CKz/em97e+/9IVT1cVT/zXrcGlgO1gEFR1tkVyAWa485Kr4sXsKreA8wA+nrb6xuh2AigOnAc0AG370ITZWtgCVATGAy8ICISY7NX4fbP74BGuGoSgH8DV4eUuxD4SVW/jLKe2t426+D29SgROSFG2epe2euBkSJyVJSyTb33E64DcBJwvoh0Ae4GugHZuH34ulfuY+BM73lLYC0HPtfTgSWq+nMRj7UbcV9AzXCf7x8ilLkS97n8BqjkrQ8iHFMi0tb7con2KKxeeQj4ADgKqIs7FvC+RN7AfXY1ge+AM8LiCT1mrwZuAj7zYjgyQvyxRPz8VHUWsA04O2w/vBbyehFwahG3V+os0ReRqm5U1Ymqul1Vf8Ulxg5hxV5W1W9UdRtwH3C5d1ZwNTBFVaeo6j5V/RDIwyWdeC4HBqnqz6q6Cu/LI4o9wPEiUlNVt6rq53HWvUZVR6hqgaruiFLmMW/bPwDDgZ4JxByTt096AANU9VdVXQE8AVwTUmylqj6nqnuBMbgvz1oxVvuUqq5S1Z9xn01hnK/gfpUc4b2+Bng5Toj3qeouVf0YlzQvj1JuDzBQVfeo6hRgKxDtS+FI4NcI0x9U1W3e/r8JeERVF3n13v8Amnln9Z8BDUWkBi7JvgDUEZHDccdh4a+Fohxrl+NODFar6ibg0QhlXlLVpV5843FfChGp6kxVPTLGo/DX4h7cCdMxqrozZPqFwAJVnaCqe3DH29qwzSRyzCYq1uf3Ot4xJCLVvNheD1n2V9yXRFqzRF9EIlJVRJ4VV83wC+6n65GhP++AVSHPVwIVcWcm9YHuoWc3QFtc8ornmAjrjeZ63NnsYhGZLSIXx1n3qjjzw8us9OIpqZq4fRP6XlbizqwK7f8DV9Xt3tNYF3Mjxqmqa4BPgEtF5EjcGWysC86bvC/qQ9YVwcawC5HbY8S4CagWJ+76wJMhx8jPgAB1vKSWh0vq7XGJ/VPcGW9ooi/KsRZ+bEU6HkITbaz3VxT9cO/rC3Gtxgp/JR4Uj7qeF8NjSuSYTVSsz+81oJuIVMb9wpqrqqHHazVgSxJjSQm78FN0f8V927dW1bUi0gz4EnfAFjo25Hk93BnDBtzB+bKq3liM7f7krXdByHojUtVvgZ7i6rq7ARO8M8BoXZUm0oVp+LbXeM+3AVVDytUuwro3cOCsbmHIuiNeqExQ+L5fE/J6DHAD7rj/LNoFUc9RInJYSLKvB3xTgrgKzcd9CYcL3U+rcL/eon0RfYyrTjgNmO29Ph9oxYE686Icaz/hqk4KHRutYJy4ARCRdriLlNFcoKozVHUtrtoIrzpnqohM58CxXrg+iRBT+HZT0g2vqi4UkZW4E4Pwahtw1W1fpWLbyWRn9LFVFJGskEcF3Df4DtzFp6NxdefhrhaRxiJSFRgITPCqHl4BLhGR870LYFnexaDwi7mRjAcGiLsYXBe4JVpBEblaRLJVdR+w2Zu8D1jv/V+cNux3ets+FrgVdwEMYB7QXkTqiUh1YEDYcvnRtuftk/HAIBGp5lVN3IHbT8X1ZxGp630294TECfAmrg76VlydfTx/F5FKXuK6GPhPCeIqNIVDq/rCPYP7rE+G/Resu4fM/xh3LWOhqu7GXRC8AfheVdd7ZYpyrI0HbhWROt6vnbuK8H4OOaa8JH54jMcM7311D4lnEy5Z78NVk50sIt28v7m/cOgJRLh8oK645qvJ9hrumGnPocdAB2J/qaUFS/SxTcEl9cLHg7j6wiq4s9HPgfciLPcyMBr3czcLd6Di1a0XXmhbjzvrupPEPoe/46oPvsddwIpVv9wJWCAiW3EXZnuo6g6v6mMQ8In3c74obbn/C8zBJfZ3cHXDeHW/43BnqnOAt8OWexK4TFzrkUjXFW7B/SpYDszE/VG9WIS4wr2G2z/LcRfxHi6c4VV7TAQa4C72xbIWl3zW4Kp4blLVxSWIq9BbwIkiErXqS1UnAY8BY73qwW9wZ5SFPsUdg4Vn7wuBnSGvi3qsPYfbZ/Nxv06n4BoR7I33Zkp4TLUEZnnH6WTgVlVdrqobgO64awUbgYa4ardYpuF+ca4VkQ1FiCERr+MS+jQvNgDEtei6EPdLMa2J2sAjphwRkfuBRqp6ddzCqYuhD9BYVW/zK4ZYxDVhfUZV6/sdSyGJcOOd30TkFuBYVe3ndyzxWB29KTe86pzrObhVT6lT1VF+bj+ciFQBzsKd1dfCVUdO8jWoMkBVR/gdQ6Ks6saUCyJyI6764l1VnR6vfDkjuKrBTbiqm0XA/b5GVEJeK56tER5XxV86eKzqxhhjAs7O6I0xJuDSro6+Zs2ampOT43cYxhhTpsyZM2eDqkbsTiXtEn1OTg55eXl+h2GMMWWKd2NXRFZ1Y4wxAWeJ3hhjAs4SvTHGBFza1dEbY4Jlz549rF69mp07d/odSiBkZWVRt25dKlasmPAyluiNMSm1evVqqlWrRk5ODhJzzBgTj6qyceNGVq9eTYMGDRJezqpujDEptXPnTmrUqGFJPglEhBo1ahT515ElemNMylmST57i7EtL9MYYE3CW6I0xJglWrFjBa68dGIBq9OjR9O3bt9jr++ijj7j44nijgCbGEr0xJVA7pzYiEvFROyfeoEgmSMITfTqxRG9MCeSvzHcD4EV45K/M9zU242zbto2LLrqIU089lSZNmjBu3DhycnIYMGAAzZo1Izc3l7lz53L++efzu9/9jmeeeQZwLVzuvPNOmjRpQtOmTRk3blzM6f3792fGjBk0a9aMYcOGAbBmzRo6depEw4YN6dfvwPgkH3zwAaeffjrNmzene/fubN26FYD33nuPE088kebNm/PGG/EGQUucNa80xpSe226DefOSu85mzWD48Kiz33vvPY455hjeeecdALZs2cJdd91FvXr1mDdvHrfffju9e/fmk08+YefOnTRp0oSbbrqJN954g3nz5vHVV1+xYcMGWrZsSfv27fn0008jTn/00UcZMmQIb7/tRtMcPXo08+bN48svv6Ry5cqccMIJ3HLLLVSpUoWHH36YqVOncthhh/HYY48xdOhQ+vXrx4033si0adM4/vjjueKKK5K2i+yM3hgTaE2bNuXDDz/krrvuYsaMGVSvXh2Azp0775/funVrqlWrRnZ2NpUrV2bz5s3MnDmTnj17kpmZSa1atejQoQOzZ8+OOj2Sjh07Ur16dbKysmjcuDErV67k888/Z+HChZxxxhk0a9aMMWPGsHLlShYvXkyDBg1o2LAhIsLVVydvtMuEzuhFpBNukOdM4HlVfTRsfnvcoNmn4AainhA2/wjcAMZvqmrxr04YY8q2GGfeqdKoUSPmzp3LlClTuPfee+nYsSMAlStXBiAjI2P/88LXBQUFSdl26HozMzMpKChAVTn33HN5/fXXDyo7L9m/dELEPaMXkUxgJG4U+sZATxFpHFbsB6A3EO1KxEOEjFBvjDGlZc2aNVStWpWrr76aO++8k7lz5ya0XLt27Rg3bhx79+5l/fr1TJ8+nVatWkWdXq1aNX799de4623Tpg2ffPIJy5YtA9w1hKVLl3LiiSeyYsUKvvvuO4BDvghKIpEz+lbAMlVdDiAiY4EuuDN0AFR1hTdvX/jCItICN+Dwe0BuyUM2xpjEff3119x5551kZGRQsWJFnn76aS677LK4y3Xt2pXPPvuMU089FRFh8ODB1K5dO+r0GjVqkJmZyamnnkrv3r056qijIq43Ozub0aNH07NnT3bt2gXAww8/TKNGjRg1ahQXXXQRVatWpV27dgl9cSQi7pixInIZ0ElVb/BeXwO0jlQFIyKjgbcLq25EJAOYBlwNnAPkRlmuD9AHoF69ei1Wrozaf74xaUVEXCubiDNdC43ybtGiRZx00kl+hxEokfapiMxR1Ygn06m+GHszMEVVV8cqpKqjVDVXVXOzsyOOhGWMMaaYEqm6+RE4NuR1XW9aIk4H2onIzcDhQCUR2aqq/YsWpjHGmOJKJNHPBhqKSANcgu8BXJnIylX1qsLnItIbV3VjSd4YY0pR3KobVS0A+gLvA4uA8aq6QEQGikhnABFpKSKrge7AsyKyIJVBG2OMSVxC7ehVdQowJWza/SHPZ+OqdGKtYzQwusgRGmOMKRG7M9YYYwLOEr0xplTF6vGzOI9k9RJ64YUXsnnz5phl7r//fqZOnVqs9Sez2+Gisk7NjDGlan+Pn8lan5Ssl1BVRVWZMmVK3LIDBw4s0bb8Ymf0xpjAGzp0KE2aNKFJkyYMHz6cFStWcMIJJ3DttdfSpEkTVq1aRU5ODhs2bADgoYce4oQTTqBt27b07NmTIUOGANC7d28mTHBdeeXk5PDAAw/QvHlzmjZtyuLFiwH44osvOP300znttNP4/e9/z5IlS/x50yHsjN4YE2hz5szhpZdeYtasWagqrVu3pkOHDnz77beMGTOGNm3aHFR+9uzZTJw4ka+++oo9e/bQvHlzWrRoEXHdNWvWZO7cufzrX/9iyJAhPP/885x44onMmDGDChUqMHXqVO6++24mTpxYGm81Kkv0xphAmzlzJl27duWwww4DoFu3bsyYMYP69esfkuQBPvnkE7p06UJWVhZZWVlccsklUdfdrVs3AFq0aLF/oJAtW7bQq1cvvv32W0SEPXv2pOBdFY1V3RhjyqXCxF8Shd0QF3ZBDHDfffdx1lln8c033/DWW2+xc+fOEm+npCzRG2MCrV27drz55pts376dbdu2MWnSJNq1axe1/BlnnLE/QW/dunX/iFGJ2rJlC3Xq1AHcKFPpwBK9MaZU1apfC4SkPWrVrxVze82bN6d37960atWK1q1bc8MNN0TtQhigZcuWdO7cmVNOOYULLriApk2b7h+VKhH9+vVjwIABnHbaaUkbwKSk4nZTXNpyc3M1Ly/P7zCMSYh1UxxfWeymeOvWrRx++OFs376d9u3bM2rUKJo3b+53WPsVtZtiuxhrjDFh+vTpw8KFC9m5cye9evVKqyRfHJbojTEmzGuvRRsVtWyyOnpjTMpZFVbyFGdfWqI3xqRUVlYWGzdutGSfBKrKxo0bycrKKtJyVnVjjEmpunXrsnr1atavX+93KIGQlZVF3boxe4U/hCV6Y0xKVaxYkQYNGvgdRrlmVTfGGBNwluiNMSbgLNEbY0zAWaI3xpiASyjRi0gnEVkiIstEpH+E+e1FZK6IFIjIZSHTm4nIZyKyQETmi8gVyQzeGGNMfHETvYhkAiOBC4DGQE8RaRxW7AegNxB+O9l24FpVPRnoBAwXkSNLGLMxxpgiSKR5ZStgmaouBxCRsUAXYGFhAVVd4c3bF7qgqi4Neb5GRNYB2cDmkgZujDEmMYlU3dQBVoW8Xu1NKxIRaQVUAr6LMK+PiOSJSJ7dVGGMMclVKhdjReS3wMvAH1V1X/h8VR2lqrmqmpudnV0aIRljTLmRSKL/ETg25HVdb1pCROQI4B3gHlX9vGjhGWOMKalEEv1soKGINBCRSkAPYHIiK/fKTwL+raoTih+mMcaY4oqb6FW1AOgLvA8sAsar6gIRGSginQFEpKWIrAa6A8+KyAJv8cuB9kBvEZnnPZql4o0YY4yJzIYSNKYEbChBky5iDSVod8YaY0zAWaI3xpiAs0RvjDEBZ4neGGMCzhK9McYEnCV6Y4wJOEv0xhgTcJbojTEm4CzRG2NMwFmiN8aYgLNEb4wxAWeJ3hhjAs4SvTHGBJwlemOMCThL9MYYE3CW6I0xJuAs0RtjTMBZojfGmICzRG+MMQGXUKIXkU4iskRElolI/wjz24vIXBEpEJHLwub1EpFvvUevZAVujDEmMXETvYhkAiOBC4DGQE8RaRxW7AegN/Ba2LJHAw8ArYFWwAMiclTJwzbGGJOoRM7oWwHLVHW5qu4GxgJdQguo6gpVnQ/sC1v2fOBDVf1ZVTcBHwKdkhC3McaYBCWS6OsAq0Jer/amJSKhZUWkj4jkiUje+vXrE1y1McaYRKTFxVhVHaWquaqam52d7Xc4xhgTKIkk+h+BY0Ne1/WmJaIkyxpjjEmCRBL9bKChiDQQkUpAD2Bygut/HzhPRI7yLsKe500zxhhTSuImelUtAPriEvQiYLyqLhCRgSLSGUBEWorIaqA78KyILPCW/Rl4CPdlMRsY6E0zxhhTSkRV/Y7hILm5uZqXl+d3GMYkREQg2p+QQLr9fZngEpE5qpobaV5aXIw1xhiTOpbojTEm4CzRG2NMwFmiN8aYgLNEb4wxAWeJ3hhjAs4SvTHGBJwlemOMCThL9MYYE3CW6I0xJuAs0RtjTMBZojfGmICzRG+MMQFnid4YYwLOEr0xJq3VzqmNiER81M6p7Xd4ZUIFvwMwxphY8lfmR+3zP1/ySzeYMsrO6I0xJuAs0RtjTMBZojfGmICzRG+MMQGXUKIXkU4iskRElolI/wjzK4vIOG/+LBHJ8aZXFJExIvK1iCwSkQFJjt8YY0wccRO9iGQCI4ELgMZATxFpHFbsemCTqh4PDAMe86Z3ByqralOgBfCnwi8BY4wxpSORM/pWwDJVXa6qu4GxQJewMl2AMd7zCUBHERFco6jDRKQCUAXYDfySlMiNMcYkJJFEXwdYFfJ6tTctYhlVLQC2ADVwSX8b8BPwAzBEVX8O34CI9BGRPBHJW79+fZHfhDHGmOhSfTG2FbAXOAZoAPxVRI4LL6Sqo1Q1V1Vzs7OzUxySMSYwKmN3zSYgkUT/I3BsyOu63rSIZbxqmurARuBK4D1V3aOq64BPgNySBm2MMQDswlUQR3jkr7S7ZgslkuhnAw1FpIGIVAJ6AJPDykwGennPLwOmqariqmvOBhCRw4A2wOJkBG6MMSYxcRO9V+feF3gfWASMV9UFIjJQRDp7xV4AaojIMuAOoLAJ5kjgcBFZgPvCeElV5yf7TRhjjIlO3Il3+sjNzdW8vDy/wzAmISIStcMtBNLt76ssirePbf87IjJHVSNWjdudscYYE3CW6I0xJuAs0RtjTMBZojfGmICzRG+MMQFnid4YYwLOEr0xxgScJXpjjAk4S/TGGBNwluiNMSbgLNEbY0zAWaI3xpiAs0RvjDEBZ4neGFNitXNqRx3pyUZ78l8FvwMwxpR9+Svzo3cXDOSLjfbkJzujN8aYgLNEb4wxAWeJ3hhjAs4SvTEmmCpjF4g9CSV6EekkIktEZJmI9I8wv7KIjPPmzxKRnJB5p4jIZyKyQES+FpGsJMZvjDGR7cJdII7yyF9Zfi4Qx030IpIJjAQuABoDPUWkcVix64FNqno8MAx4zFu2AvAKcJOqngycCexJWvTGpImKu6HuKr+jMCayRM7oWwHLVHW5qu4GxgJdwsp0AcZ4zycAHUVEgPOA+ar6FYCqblTVvckJ3Zj0UHM9fNwBlh8H7T/2OxpjDpVIoq8DhJ6rrPamRSyjqgXAFqAG0AhQEXlfROaKSL9IGxCRPiKSJyJ569evL+p7MMY3JwCft4Fm8+DHOjDhMqi30u+ojDlYqi/GVgDaAld5/3cVkY7hhVR1lKrmqmpudnZ2ikMyJkk+/phPgcO3wpkfwXkfQMU98OYfoOo2n2MzJkQiif5H4NiQ13W9aRHLePXy1YGNuLP/6aq6QVW3A1OA5iUN2hjfvfwynHsua4E2n8MXreHbRtBjLJwyH168zu8AjTkgkUQ/G2goIg1EpBLQA5gcVmYy0Mt7fhkwTVUVeB9oKiJVvS+ADsDC5IRujA9U4e9/h2uvhbZtOQNY0eDA7Pc7Qf9H4YrxcEjzNGN8EjfRe3XufXFJexEwXlUXiMhAEensFXsBqCEiy4A78I5xVd0EDMV9WcwD5qrqO0l/F8aUht27oXdvePBB6NUL3nuPzRGKDfkbvHolDAJ4++3SjNCYiMSdeKeP3NxczcvL8zsMYw62bx9cdBG89x4MHAj33gvejTeROvPK2gEzq0KLatVg1iw46aTSj7kURdsPBwpAcXNNzHULxZtXwpjSkYjMUdXcSPPszlhjEvHccy7JjxgB990HIjGL76wCfwCoUgW6dIHNm0shSGMis0RvTDw//QR33QVnnQV//nPCi60GmDgRVqyAK6+EvXYLifGHJXpj4rntNti5E555Ju6Z/CHatoWRI+Hdd+HRR1MSnjHxWKI3JpYpU2D8eFcn36hR8dZx443QvTs89BAsW5bc+Mx+v8mHCtbBSkSW6I2JZutWuPlmaNwY+kW8qTtxw4dD5cpufQG6AOinzAJ3F+ajd8E3J0N+bVjaCK58FWSf39GlF0v0xkTzwAOwciVtFy5EKleO2NVtwo45BgYNgg8/hLFjUxdzCsUaF7a0HLkJerwOr1wF634DM4A7hsJPv4V7HobNR8KrV8OXp7leGGO2uilPVDWtHi1atFBjfDdnjmpGhj4DGvNfrPlw8DoLClRzc1Vr1VLdtMmXt1US8d5rvP1U0u1eMwbdXQFV0HU10Zd6oZeCHrH5wHZkL9rjNXTZca7cR+3R1p8lP6Z0BORplLxqZ/TGhCsogD59IDs7uXe3ZmbCs8/C+vUwYEAy1xx4ubNhVB/45Axo8xnUXgt/HA0TgV+qHyinGTC2J5y0CG4GTlgCn58Ob3SFExf5FHwasERvTLinnoI5c+DJJyPe+VoizZvDX/7iEv7nnyd77YFUE5h4KeTXgssmwKw2sC8z9jJ7KsHTwPHL4N6H4OxpMLc5dP5vaUScfuzOWGNC/fCDu/jaoQO8/TaSkRH37spYd2ZG/Pv69Ve3jaOPhrw8qFgxCYGnXrHvUPXmFyvXFBQwtWJF2laGMz6BuS2KsN2QednrYHJnaDkbbnoGnr+xBDGlKbsz1phEqELfvu7/kSOL3mY+UdWqwT//CfPnu9Y4Jrp77uEc4P+ejpDki2D9b6Dj/+D98+G5Pu4sn0rRx5QN3Hiy0Srv/XrYxVjjm3feUQXVIUP2TyKZF2ND7dunesklqlWrqq5YUQpvruTivdekX4ydMEEV9F/F3W6EeRV2o6OvdRdqnwLNKAjOhVrsYqwx0RU2G5x+0UWsACr+7W+pbzYo4vrNgQO/IswBixa5nkLbtOG2JK62oCL0Hg2P9YM/A2N7QOWdSdxAmrJEb8q9/JX5tPkU2gNDn4QCdy6Y+jbY9eu7vu3ffhsmTUrxxsqQX36Brl2halWYMIHdyV6/QP/H4Hag+wR49wI4YkuyN5JeLNEbA9z5OGw8Gl64vpQ3fOutcOqpB/rTKe9U3Zn8smWu64k64cNTJ89w4KpXoO1MN7h7zQAPV22J3pR7jYA/vAn/uhm2H1bKG69YEYYOhVWr4OmnS3njaWjQIPfrZsgQ1/IpxV67Ci56x7W7H3Z7yjfnG0v0ptz7K7C7Eoy4xacAzj4bOnaEf/zDNb0sj1Rdp2/33QdXXeV+6ZSSD8+Dwf3g6leh3fRS22ypskRvyre1a7kWeOmPrgmeb/7xD9iwwZ3dlzeq8Le/wf33u7F4R49OXdPWKB4ZACvrwVN9XWdpQWOJ3pRvI0ZQCXjirz7H0aqVuwD5xBMu4ZcXe/fCDTe4L7hbboGXXoIKFUo9jB1V4fZhcMrXrs1+0FiiN+XXr7/Cv/7FROC74/0OBnj4Ydi2rfwMULJrF1xxBbz4ojubf/JJyPAvJU3qCh+cCw/dB9m+RZEaCe1VEekkIktEZJmIHNLPk4hUFpFx3vxZIpITNr+eiGwVkb8lKW5jSu7552HzZh73O45CjRvDNde4vnZWr/Y7mtTatg06d3ZDLQ4d6pqZlnJ1zSEEbhkBVbdD0L5q4yZ6EckERuK6d24M9BSRxmHFrgc2qerxwDDgsbD5Q4F3Sx6uMUmyZw8MGwYdOjDb71hCPfgg7NsHAwf6HUnqbN4M550HU6fCCy/A7enT3GXpCa71zXUQqE7nEjmjbwUsU9XlqrobGAt0CSvTBRjjPZ8AdBTvtkIR+QPwPbAgKREbkwzjxrkmjSUdOSqWytH7Usk8LDPyvAYNGLFnDwXPPQfffpu62Pzy6adw5pkwe7ZrJ3/ddX5HdIiH74Ufwd2xHJAB3RNJ9HWAVSGvV3vTIpZR1QJgC1BDRA4H7gL+HmsDItJHRPJEJG/9+gDftWDSgyoMHgxNmsAFF6RuO7s4cIdt2GPf9n1R5z281i3K/fenLrZSlLEXugH8/vdwxhmuh9C334ZLL/U7tIi2VnNNbpkzx1XvBUCqr3w8CAxT1a2xCqnqKFXNVdXc7OygXQYxaef99+Hrr12TPr/rhSNYV8vdtcnYsTBvnr/BlMBhW6HvCDeO60SA/HzXv88PP7iqmzQ2DtwNW3ffDRs3+h1OiSWS6H8Ejg15XdebFrGMiFQAqgMbgdbAYBFZAdwG3C0ifUsWsjElNHiwu7W+Z0+/I4nqcYAjj4R77vE5kqLLXgeD7oYf6sGIv7gBQy4FWLrUVYccfrjfISbmqadgy5Yy+RmESyTRzwYaikgDEakE9AAmh5WZDPTynl8GTPN6zmynqjmqmoM7SfmHqj6VnNCNKYa8PPh//89dAKxUye9ootoCcNddMGUKzJzpdzgJO2khzGkB/R+FaWfD6Z/CGZ/CG+CGUixLmjRxbftHjXLVOGVY3ETv1bn3Bd4HFgHjVXWBiAwUkc5esRdwdfLLgDsguUNtGpM0gwdD9epw441+RxLfX/4CtWu78WXLQDfGp38KM9tChQLIzXM9Q35+ut9RldCDD0J2tut0riyL1lG9Xw8beMSkzNKlqhkZqv37HzSZEg6okfR5oQNfjBzpBkOZMMGHHXawWDFfAro9C13SEM1ZntyBPFK5j+Puf1XVf/7TfQYzZyZhL6YONvCIMcDjj7veIsvS2dmNN7oBxW+6Cdau9TuaiK57ASYBXzd147quaOB3REl23XVQowY8Fn57UNlhid6UD2vWwJgx7o+2Vi2/o0lcxYrw8suuu4Ybb0yvKhyFuwfBCzfAh8DZ02BDEBvNHXaYq6t/6y1YUDZvB7JEb8qHYcPczS933ul3JEXXuDE88ohre/7ii35HA7i28SNugUH3wr+vgUuAbWWkMU2x9O3rRrwaMsTvSIrFEr0Jvk2b4JlnXAdaDcpovcKtt8JZZ7lqp+XLfQ2l0i54vSf0HQmD73RjsAawZ9+D1agB118Pr75aJvshskRvgm/kSNi6FfqX4cZgGRmun/aMDOjVy7db86tug7cugcv/A38dAncNBi0vWeSOO1w/RMOG+R1JkZWXj8iUV9u3u+5vL7oImjb1O5qSqVfP3Vk6c6brt76UVQc+OA86/g96vwRD/e7Dv7Tl5ECPHq5d/aZNfkdTJJboTbC98IIbyKMsn82HuuYa6NbNDbk3f37pbTc/n4+AlrOh+39gTO/S23Ra6dfP/TosY+P7iqbTVXwgNzdX8/Ly/A7DBMGePXD88e5MeMaMqMVExHUoFnEm0efFm1/ced78qH+bGza4uzZr1YIvvoDKlWOsKAl++AHOOYdt335L1/fdGKuHKMn7iaPYn08JP7uo8V5wAcydCytWQJUqMTZQukRkjqrmRppnZ/QmuF5/3SWpoJzNF6pZ0/1SmT8fHnggtdtasgTatoV16ziXKEk+iGJ0MX3me+/BunXw73/7HWXCLNGbYNq3z93g0rQpXHih39Ek30UXuXb1gwfH/LVSIvPmQbt2bsi/jz7is9RsJT3F6GL6430wC1xTyzLSX70lehNMb70FCxe6s/k07Io4KYYOheOOc1UJL7yQvJupfv3Vtdc/80zIynJfJM2aJWfdQSAwGGDZMnjjDb+jSYglehM8qu4GowYN4PLL/Y4mdQ4/HD76CFq3hhtucAN5FLfv9L173dB+11zjOlK7/nrXymTmTGjUKJlRB8KbAA0bul+NaXadMxJL9CZ4pk+HWbPcXbAVKvgdTWrVrQsffuj68Xn7bVdV9eGHiS+/eLHrHTMnB849ly2vvsoz27fze0C++gqpX39/3bQ5YB+442vOHNftdZqzVjcmeDp1cvXLK1a4qoc4ylyrm2jmzYMrr4RFi9wdtI88cuj737DBnaVPn+4S1Lx5rp/488+HXr3IuuIKdqXL+ylc1KdWN3Hfz44d7lfjKae4Uct8FqvVTcBPd0y58+WX7o8uUpILumbN3Blmv34wfDj8739ulKQ1a1xinz79QKdcWVmuymfIELjqKlddA+y64grfwi9zsrJc1xQDBrjBzlu29DuiqOyM3gRLjx7w7ruuWWX16gktEpgz+lBTpsAf/+iaAYKrz2/bFtq3d4/c3Ijt71N59hy4M3pV+OUXd6/GySfDtGm+Xvi3M3pP7Zza5K/Mjzq/Vv1arF2Rnn1+mwR89x385z+u7jTBJB9YF17oBkB/5x1Xb9+sWfCvV/jhiCPcKFR//rPb1xdf7HdEEZWrM/qYZwZQ8rMo46//+z/XLHDFCvjtbxNeLJBn9MVkZ/SJL7v//ezZ4+5Uzsx0N7H59IVqd8aa4Fu7Fl56CXr3LlKSN6bEKlZ0zSwXLXL3M6ShhBK9iHQSkSUiskxEDrmfXEQqi8g4b/4sEcnxpp8rInNE5Gvv/7OTHL8xzj//6c6syuLAIqbs69LFXQN54AF3w1maiZvoRSQTGAlcADQGeopI47Bi1wObVPV4YBhQOLjiBuASVW0K9AJeTlbgKRGjf4vaObX9js5Es2ULvzz6KOP37UMaNgzGZ2fHYnoL/3wyMmg9cybk5/PEsen3izKRM/pWwDJVXa6qu4GxQJewMl2AMd7zCUBHERFV/VJV13jTFwBVRCTFXe2VQIz+LWJdxDU+e/ZZjlDlsTyC89nZsZjeInw+XyiMuxxu2rLNNWlNI4kk+jrAqpDXq71pEcuoagGwBagRVuZSYK6q7grfgIj0EZE8Eclbv359orEbAzt3wrBhfADMbeF3MKa8G/AIVAS4/36/QzlIqVyMFZGTcdU5f4o0X1VHqWququZmZwdxGHmTMi+/DGvX7q8rNMZP3x8HT4FrGPDNN36Hs18iif5H4NiQ13W9aRHLiEgF3KhjG73XdYFJwLWq+l1JAzZmv717XTe9ublM8zsWYzwPg2tf36+f36Hsl0iinw00FJEGIlIJ6AFMDiszGXexFeAyYJqqqogcCbwD9FfVT5IUszHOpEmuq9i77vI7EmP22wRwzz3uDu2pU/0OB0jwhikRuRAYDmQCL6rqIBEZCOSp6mQRycK1qDkN+BnooarLReReYADwbcjqzlPVddG2VdIbpuLd/ZqSocVM6VN1fYv88gssWoRUqBD9s8vCXTyLub4o0328YcqPY9FumErOsrpjB5x4Ihx1lOt/KCP1teSxbpgK3J2xqbzLMd32Vbn2v//BOefAc8/BDTf4c3drSZZN02PREn1yllVVeO0112Fc4Y18KWZ3xprgefRRdwfsNdf4HYkxkfXoAaefDrfc4rpG8JElelP2zJnj6j5vvz1iD4zGpIWMDNfJ3hFHwCWXQL5/9z9Yojdly44drqfA6tXhTxFb6xqTPurUgcmTYf16+MMf3H0fPiiX/ZZW3QZnT4Oq22HTUSEPYPNe2Jfpd4QmIlU3lumsWW5Q5iOO8DsiY+Jr0QJeecWN6Xv99e55KfdbX24S/RFb4GLg0m7Q6T2ouiNKwQqw5Qj4vgGM6eUem44uzUhNVA89BK+/7urnu3b1OxpjEtetGwwa5JpdnnQS3HtvqW4+0K1uaq6HLv+Fbm/AOVOh0h748RiY1BXe6Ab5teCoTSGPXnDUA+5561nQZhbsrAzjL4dnx8KneyJv0wYsSb0/ZR/Jsxu2MBr4Y7RCadj6IiXrjdNcNKNqBvu27yvyvP2s1U1qWkWpwrXXwiuv0B3XKVi4kuSSWK1uUNW0erRo0UJLAlAU7fUSWpCBKuiy49DBf0Nbg8peNz/iPw5+fco8dOT/oVuqufV8fTLa959o9U2HLmdSaNYs3Q76cTu00s7EPruE5/m1bDrGlOL3U1xlcR9HtWOHzgTdVgVtMTvp+ylPo+TVwJ7RN1oCV70KEy+F+afgvoGL+S192FboUQ3+lAst82B7FRh+Gwy6B7YfhrWxT6VVq6BlS5bn59NqPWysGaVcmp7ZlamYSrKsndEfNC/We/2NCF/Uh0q7oeVsWFMn8WVjKZft6JeeAA8MhPmn4j6UEth2OLwAtJoNzee4qp+7H4FFJ8GlkX5/meTYutU1S9uxg4uJkeSNKUPWA5e8BdV+hcmdXeOQVAtsok+VL5vD1a9C2xmupc6E7vABwOLFfocWLPv2wdVXuwGux41jkd/xGJNE3zSFHmMhYx8cuTn127NEX0yftIUWc6DvCGgJ0LSp660uDYcRK3NUXUdl//0vDB8OnTr5HZExSTflIsjNC6u6SRFL9CWwtwKM7AuNwF1Nf/xx15HR2LEuWZmi27bN9Q8yZAjcfDP07et3RMakTGnds2OJPgnWgxv9/fPPXf8rPXvC2Wen1cADZcLSpdC6NYwb59ocjxhR6jeWGBNEluiTqXVrd9fmM8+4ToyaNYM77oAtW/yOLP1NmgS5ua4/kPffh7vvLpWuXY0pD+wvKdkyM10fLEuXwg03uDrmE05wQ95Zdc6hCgpcfXy3bq7aa84c1/2wMSZpLNGnSo0a7sz+iy+gfn1Xh9++PXz1ld+RpY/8fDjvPDcc4E03wYwZUK+e31EZEziW6FMtNxc++wyef941wWzeHC6/HGbP9jsy/+zY4X7hNG/u9s2YMfD009blsDEpYom+NGRkuF7rli51TTA/+ABatYKzznLjSpaXKp3Fi10f8nXquF84Rx7pEv211/odmTGBZom+NB11FDzyiLut/4kn3MDWF14Ip5wC//437N7td4TJt2uXG1KtQwfXa9/IkXDuuW4owG++cResjTEpZYneD9WqudY4333nEjxAr15w3HGu+9KPP3YJsqz65Rd46y03hFqdOq5d/OrVrnvh1atd88mzz7amk8aUlmi9nYU+gE7AEmAZ0D/C/MrAOG/+LCAnZN4Ab/oS4Px420pW75Wl2ptdSXuv3LdPdcoU1Y4dVTMzVUG1ShXV889Xffxx1S+/VN27t2TbSKVdu1SnT1e9/37V3//+4Pdw6aWqH3xQ4vhT9tn5tWw6xpTi95OSzz5N91NJ3k8J9lPxe68UkUxgKXAusBqYDfRU1YUhZW4GTlHVm0SkB9BVVa8QkcbA60Ar4BhgKtBIVfdG214y+6M/dCYp67kv3n5M2JYt7ox+6lT3WOT18lKzJrRr51ql1K7tbswKfdSokboz5L17Yd06+Omngx9r17pfJTNnujtaMzKgZUvXPLJjRzcwclZWUkKI+blC2vZiWKZiKsmy1nvlQfNivdd476cE+ylq75WJjDDVClimqsu9lY0FugALQ8p0AR70nk8AnhIR8aaPVdVdwPcissxb32fFeSPlQvXq0LmzewD8+CNMm+aS/qxZ7v9I/elUqJC0pHqI7dtdJ2Phjj4a6taF3r1dcj/zTHeB1RiTVhJJ9HWAVSGvVwOto5VR1QIR2QLU8KZ/HrbsIV34iEgfoI/3cquILEko+mhindgePK8msKGYyx48y+/65oIC161vahy6nwB+/tk95s93F1lTLd4uLuZnl8Rlk3Y8lcll46w37G8k8jFVyjGlatm4+SDxZYuyn+pHm5EWY8aq6ihgVGlvV0Tyov3UMQfYfkqM7afE2b5KTLL2UyKtbn4Ejg15XdebFrGMiFQAqgMbE1zWGGNMCiWS6GcDDUWkgYhUAnoAk8PKTAZ6ec8vA6Z5V4EnAz1EpLKINAAaAl8kJ3RjjDGJiFt149W59wXeBzKBF1V1gYgMxDXnmYwbae9l72Lrz7gvA7xy43EXbguAP8dqceODUq8uKqNsPyXG9lPibF8lJin7Ke0GBzfGGJNcdmesMcYEnCV6Y4wJuHKX6EXkcRFZLCLzRWSSiBwZMm+AiCwTkSUicr6PYaYFEekuIgtEZJ+I5IbNs30VQkQ6eftimYj09zuedCEiL4rIOhH5JmTa0SLyoYh86/1/lJ8xpgMROVZE/p+ILPT+5m71pidlX5W7RA98CDRR1VNwXTsMAPC6a+gBnIzr2+dfXvcP5dk3QDdgeuhE21cH8977SOACoDHQ09tHBkbjjpFQ/YH/qWpD4H/e6/KuAPirqjYG2gB/9o6hpOyrcpfoVfUDVS3wXn6Oa9sPId01qOr3uI7YWvkRY7pQ1UWqGukuZdtXB9vfTYiq7gYKuwkp91R1Oq4lXqguwBjv+RjgD6UZUzpS1Z9Uda73/FdgEa4XgaTsq3KX6MNcB7zrPY/U1cMh3TUYwPZVONsfRVNLVX/ynq8FavkZTLoRkRzgNFxPwEnZV2nRBUKyichUoHaEWfeo6n+9Mvfgfi69WpqxpZtE9pUxqaKqKiLWxtsjIocDE4HbVPWX0H5vSrKvApnoVfWcWPNFpDdwMdBRD9xIUC67a4i3r6Iol/sqBtsfRZMvIr9V1Z9E5LfAOr8DSgciUhGX5F9V1Te8yUnZV+Wu6kZEOgH9gM6quj1klnXXkDjbVwdLpJsQc0Bolym9gHL/y9Hr1v0FYJGqDg2ZlZR9Ve7ujPW6aaiM63QN4HNVvcmbdw+u3r4A99Pp3chrKR9EpCswAsgGNgPzVPV8b57tqxAiciEwnAPdhAzyN6L0ICKvA2fiutvNBx4A3gTGA/WAlcDlqhp+wbZcEZG2wAzga6Bw8Ie7cfX0Jd5X5S7RG2NMeVPuqm6MMaa8sURvjDEBZ4neGGMCzhK9McYEnCV6Y4wJOEv0xhgTcJbojTEm4P4/nesSPOKjGroAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "labels = np.concatenate([np.random.normal(-20, 1, 10), np.random.normal(0, 2, 100), np.random.normal(12, 2, 300)], -1)\n",
    "labels[(-1<labels) & (labels<1)] = 0   # This is done to create some 'gaps' for demo purposes\n",
    "labels[(10<labels) & (labels<12)] = 0  # This is done to create some 'gaps' for demo purposes\n",
    "\n",
    "n_bins = 50\n",
    "label_range=None\n",
    "reweight = 'inv'\n",
    "lds_kernel='gaussian'\n",
    "lds_ks=5\n",
    "lds_sigma=2\n",
    "\n",
    "weights_per_sample = prepare_LDS_weights(labels, n_bins, label_range=label_range, reweight=reweight, \n",
    "                                         lds_kernel=lds_kernel, lds_ks=lds_ks, lds_sigma=lds_sigma, show_plot=True)\n",
    "\n",
    "n_bins = 50\n",
    "label_range=None\n",
    "reweight = 'sqrt_inv'\n",
    "lds_kernel='gaussian'\n",
    "lds_ks=5\n",
    "lds_sigma=2\n",
    "\n",
    "weights_per_sample = prepare_LDS_weights(labels, n_bins, label_range=label_range, reweight=reweight, \n",
    "                                         lds_kernel=lds_kernel, lds_ks=lds_ks, lds_sigma=lds_sigma, show_plot=True)\n",
    "\n",
    "n_bins = None\n",
    "label_range=None\n",
    "reweight = 'sqrt_inv'\n",
    "lds_kernel='triang'\n",
    "lds_ks=9\n",
    "lds_sigma=1\n",
    "\n",
    "weights_per_sample = prepare_LDS_weights(labels, n_bins, label_range=label_range, reweight=reweight, \n",
    "                                         lds_kernel=lds_kernel, lds_ks=lds_ks, lds_sigma=lds_sigma, show_plot=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This loss will allow you to pass a different weight per individual sample."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# export\n",
    "class WeightedPerSampleLoss(Callback):\n",
    "    order = 65\n",
    "\n",
    "    r\"\"\"Loss wrapper than applies a weight per sample during training\n",
    "\n",
    "    Weights are not applied to the validation loss.\n",
    "\n",
    "    Args:\n",
    "        instance_weights:   weights that will be applied. Weights will be normalized to 1.\n",
    "                            You can pass weights for the entire dataset or just for the training set.\n",
    "    \"\"\"\n",
    "\n",
    "    def __init__(self, instance_weights):\n",
    "        store_attr()\n",
    "\n",
    "    def before_fit(self):\n",
    "        self.old_loss = self.learn.loss_func\n",
    "        self.reduction = getattr(self.learn.loss_func, 'reduction', None)\n",
    "        self.learn.loss_func = _PerInstanceLoss(crit=self.learn.loss_func)\n",
    "        if len(self.instance_weights) == len(self.learn.dls.train.dataset):\n",
    "            self.instance_weights = torch.cat([self.instance_weights, torch.zeros(len(self.learn.dls.valid.dataset))])\n",
    "        assert len(self.instance_weights) == len(self.learn.dls.train.dataset) + len(self.learn.dls.valid.dataset)\n",
    "        self.instance_weights = self.instance_weights / torch.sum(self.instance_weights) * len(self.instance_weights)\n",
    "        self.instance_weights = torch.as_tensor(self.instance_weights, device=self.learn.dls.device)\n",
    "\n",
    "    def before_batch(self):\n",
    "        self.learn.loss_func.training = self.training\n",
    "        if self.training:\n",
    "            input_idxs = self.learn.dls.train.input_idxs\n",
    "            self.learn.loss_func.weights = self.instance_weights[input_idxs]\n",
    "\n",
    "    def after_fit(self):\n",
    "        self.learn.loss_func = self.old_loss\n",
    "        if self.reduction is not None: self.learn.loss_func.reduction = self.reduction\n",
    "            \n",
    "\n",
    "class _PerInstanceLoss(Module):\n",
    "    def __init__(self, crit):\n",
    "        self.crit = crit\n",
    "        self.crit.reduction = 'none'\n",
    "        self.weights = None\n",
    "        self.training = False\n",
    "\n",
    "    def forward(self, input, target):\n",
    "        if not self.training:\n",
    "            return self.crit(input, target).mean()\n",
    "        else:\n",
    "            return ((self.crit(input, target) * self.weights)).mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# BatchSubsampler"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# export\n",
    "\n",
    "class BatchSubsampler(Callback):\n",
    "    \"\"\" Callback that selects a percentage of samples and/ or sequence steps with replacement from each training batch\n",
    "\n",
    "    Args:\n",
    "    ====\n",
    "\n",
    "    sample_pct:     percentage of random samples (or instances) that will be drawn. If 1. the output batch will contain the same number of samples\n",
    "                    as the input batch.\n",
    "    step_pct:       percentage of random sequence steps that will be drawn. If 1. the output batch will contain the same number of sequence steps\n",
    "                    as the input batch. If used with models that don't use a pooling layer, this must be set to 1 to keep the same dimensions.\n",
    "                    With CNNs, this value may be different.\n",
    "    same_seq_len:   If True, it ensures that the output has the same shape as the input, even if the step_pct chosen is < 1. Defaults to True.\n",
    "    update_y:       used with step_pct. If True, it applies the same random indices to y. It can only be used with sequential targets.\n",
    "    \"\"\"\n",
    "\n",
    "    def __init__(self, sample_pct:Optional[float]=None, step_pct:Optional[float]=None, same_seq_len:bool=True, update_y:bool=False):\n",
    "        store_attr()\n",
    "\n",
    "    def before_fit(self):\n",
    "        self.run = not hasattr(self, \"gather_preds\")\n",
    "        if not(self.run): return\n",
    "\n",
    "    def before_batch(self):\n",
    "        if not self.training: return\n",
    "\n",
    "        if self.sample_pct is not None:\n",
    "            B = self.x.shape[0]\n",
    "            if isinstance(self.sample_pct, tuple):\n",
    "                sample_pct = np.random.rand() * (self.sample_pct[1] - self.sample_pct[0]) + self.sample_pct[0]\n",
    "            else:\n",
    "                sample_pct = self.sample_pct\n",
    "            idxs = np.random.choice(B, round(B * sample_pct), True)\n",
    "            self.learn.xb = tuple(xbi[idxs] for xbi in self.learn.xb)\n",
    "            self.learn.yb = tuple(ybi[idxs] for ybi in self.learn.yb)\n",
    "\n",
    "        if self.step_pct is not None:\n",
    "            S = self.x.shape[-1]\n",
    "            if isinstance(self.step_pct, tuple):\n",
    "                step_pct = np.random.rand() * (self.step_pct[1] - self.step_pct[0]) + self.step_pct[0]\n",
    "            else:\n",
    "                step_pct = self.step_pct\n",
    "            if self.step_pct != 1 and self.same_seq_len:\n",
    "                idxs = np.sort(np.tile(np.random.choice(S, round(S * step_pct), True), math.ceil(1 / step_pct))[:S])\n",
    "            else:\n",
    "                idxs = np.sort(np.random.choice(S, round(S * step_pct), True))\n",
    "            self.learn.xb = tuple(xbi[...,idxs] for xbi in self.learn.xb)\n",
    "            if self.update_y:\n",
    "                self.learn.yb = tuple(ybi[...,idxs] for ybi in self.learn.yb)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# BatchLossFilter"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#export\n",
    "class BatchLossFilter(Callback):\n",
    "    \"\"\" Callback that selects the hardest samples in every batch representing a percentage of the total loss\"\"\"\n",
    "\n",
    "    def __init__(self, loss_perc=1., schedule_func:Optional[callable]=None):\n",
    "        store_attr()\n",
    "\n",
    "    def before_fit(self):\n",
    "        self.run = not hasattr(self, \"gather_preds\")\n",
    "        if not(self.run): return\n",
    "        self.crit = self.learn.loss_func\n",
    "        if hasattr(self.crit, 'reduction'): self.red = self.crit.reduction\n",
    "\n",
    "    def before_batch(self):\n",
    "        if not self.training: return\n",
    "        if self.schedule_func is None: loss_perc = self.loss_perc\n",
    "        else: loss_perc = self.loss_perc * self.schedule_func(self.pct_train)\n",
    "        if loss_perc == 1.: return\n",
    "        with torch.no_grad():\n",
    "            if hasattr(self.crit, 'reduction'):  setattr(self.crit, 'reduction', 'none')\n",
    "            losses = self.crit(self.learn.model(self.x), self.y)\n",
    "            if losses.ndim == 2: losses = losses.mean(-1)\n",
    "            if hasattr(self.crit, 'reduction'):  setattr(self.crit, 'reduction', self.red)\n",
    "            losses /= losses.sum()\n",
    "            idxs = torch.argsort(losses, descending=True)\n",
    "            cut_idx = max(1, torch.argmax((losses[idxs].cumsum(0) > loss_perc).float()))\n",
    "            idxs = idxs[:cut_idx]\n",
    "            self.learn.xb = tuple(xbi[idxs] for xbi in self.learn.xb)\n",
    "            self.learn.yb = tuple(ybi[idxs] for ybi in self.learn.yb)\n",
    "\n",
    "    def after_fit(self):\n",
    "        if hasattr(self.learn.loss_func, 'reduction'):  setattr(self.learn.loss_func, 'reduction', self.red)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# RandomWeightLossWrapper"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# export\n",
    "\n",
    "class RandomWeightLossWrapper(Callback):\n",
    "\n",
    "    def before_fit(self):\n",
    "        self.run = not hasattr(self, \"gather_preds\")\n",
    "        if not(self.run): return\n",
    "        self.crit = self.learn.loss_func\n",
    "        if hasattr(self.crit, 'reduction'): self.red = self.crit.reduction\n",
    "        self.learn.loss_func = self._random_weight_loss\n",
    "\n",
    "    def _random_weight_loss(self, input: Tensor, target: Tensor) -> Tensor:\n",
    "        if self.training:\n",
    "            setattr(self.crit, 'reduction', 'none')\n",
    "            loss = self.crit(input, target)\n",
    "            setattr(self.crit, 'reduction', self.red)\n",
    "            rw = torch.rand(input.shape[0], device=input.device)\n",
    "            rw /= rw.sum()\n",
    "            non_red_loss = loss * rw\n",
    "            return non_red_loss.sum()\n",
    "        else:\n",
    "            return self.crit(input, target)\n",
    "\n",
    "    def after_fit(self):\n",
    "        if hasattr(self.crit, 'reduction'): setattr(self.crit, 'reduction', self.red)\n",
    "        self.learn.loss_func = self.crit"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# SamplerWithReplacement"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# export\n",
    "\n",
    "class SamplerWithReplacement(Callback):\n",
    "    \"\"\" Callback that selects a percentage of samples and/ or sequence steps with replacement from each training batch\"\"\"\n",
    "\n",
    "    def before_fit(self):\n",
    "        self.run = not hasattr(self, \"gather_preds\")\n",
    "        if not(self.run): return\n",
    "\n",
    "        self.old_get_idxs = self.learn.dls.train.get_idxs\n",
    "        self.learn.dls.train.get_idxs = self._get_idxs\n",
    "\n",
    "    def _get_idxs(self):\n",
    "        dl = self.learn.dls.train\n",
    "        if dl.n==0: return []\n",
    "        if dl.weights is not None:\n",
    "            return np.random.choice(dl.n, dl.n, p=dl.weights)\n",
    "        idxs = Inf.count if dl.indexed else Inf.nones\n",
    "        if dl.n is not None: idxs = np.random.choice(dl.n,dl.n,True)\n",
    "        if dl.shuffle: idxs = dl.shuffle_fn(idxs)\n",
    "        return idxs\n",
    "\n",
    "    def after_fit(self):\n",
    "        self.learn.dls.train.get_idxs = self.old_get_idxs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# BatchMasker"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# export\n",
    "\n",
    "class BatchMasker(Callback):\n",
    "    \"\"\" Callback that applies a random mask to each sample in a training batch\n",
    "\n",
    "    Args:\n",
    "    ====\n",
    "    r:                  probability of masking.\n",
    "    subsequence_mask:   apply a mask to random subsequences.\n",
    "    lm:                 average mask len when using stateful (geometric) masking.\n",
    "    stateful:           geometric distribution is applied so that average mask length is lm.\n",
    "    sync:               all variables have the same masking.\n",
    "    variable_mask:      apply a mask to random variables. Only applicable to multivariate time series.\n",
    "    future_mask:        used to train a forecasting model.\n",
    "    schedule_func:      if a scheduler is passed, it will modify the probability of masking during training.\n",
    "    \"\"\"\n",
    "\n",
    "    def __init__(self, r:float=.15, lm:int=3, stateful:bool=True, sync:bool=False, subsequence_mask:bool=True, \n",
    "                 variable_mask:bool=False, future_mask:bool=False, schedule_func:Optional[callable]=None):\n",
    "        store_attr()\n",
    "\n",
    "    def before_fit(self):\n",
    "        self.run = not hasattr(self, \"gather_preds\")\n",
    "        if not(self.run): return\n",
    "\n",
    "    def before_batch(self):\n",
    "        if not self.training: return\n",
    "        r = self.r * self.schedule_func(self.pct_train) if self.schedule_func is not None else self.r\n",
    "        mask = create_mask(self.x,  r=r, lm=self.lm, stateful=self.stateful, sync=self.sync, \n",
    "                        subsequence_mask=self.subsequence_mask, variable_mask=self.variable_mask, future_mask=self.future_mask)\n",
    "        self.learn.xb = (self.xb[0].masked_fill(mask, 0),)\n",
    "        # In my tests, mask-based compensation doesn't seem to be important. ??\n",
    "        # mean_per_seq = (torch.max(torch.ones(1, device=mask.device), torch.sum(mask, dim=-1).unsqueeze(-1)) / mask.shape[-1])\n",
    "        # self.learn.xb = (self.xb[0].masked_fill(mask, 0) / (1 - mean_per_seq), )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# SamplerWithReplacement"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# export\n",
    "\n",
    "class SamplerWithReplacement(Callback):\n",
    "    \"\"\" Callback that modify the sampler to select a percentage of samples and/ or sequence steps with replacement from each training batch\"\"\"\n",
    "\n",
    "    def before_fit(self):\n",
    "        self.run = not hasattr(self, \"gather_preds\")\n",
    "        if not(self.run): return\n",
    "\n",
    "        self.old_get_idxs = self.learn.dls.train.get_idxs\n",
    "        self.learn.dls.train.get_idxs = self._get_idxs\n",
    "\n",
    "    def _get_idxs(self):\n",
    "        dl = self.learn.dls.train\n",
    "        if dl.n==0: return []\n",
    "        if dl.weights is not None:\n",
    "            return np.random.choice(dl.n, dl.n, p=dl.weights)\n",
    "        idxs = Inf.count if dl.indexed else Inf.nones\n",
    "        if dl.n is not None: idxs = np.random.choice(dl.n,dl.n,True)\n",
    "        if dl.shuffle: idxs = dl.shuffle_fn(idxs)\n",
    "        return idxs\n",
    "\n",
    "    def after_fit(self):\n",
    "        self.learn.dls.train.get_idxs = self.old_get_idxs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/javascript": [
       "IPython.notebook.save_checkpoint();"
      ],
      "text/plain": [
       "<IPython.core.display.Javascript object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "060_callback.core.ipynb saved at 2021-11-26 19:26:46.\n",
      "Converted 060_callback.core.ipynb.\n",
      "\n",
      "\n",
      "Correct conversion! 😃\n",
      "Total time elapsed 0.086 s\n",
      "Friday 26/11/21 19:26:49 CET\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "                <audio  controls=\"controls\" autoplay=\"autoplay\">\n",
       "                    <source src=\"data:audio/wav;base64,UklGRvQHAABXQVZFZm10IBAAAAABAAEAECcAACBOAAACABAAZGF0YdAHAAAAAPF/iPh/gOoOon6w6ayCoR2ZeyfbjobxK+F2Hs0XjKc5i3DGvzaTlEaraE+zz5uLUl9f46fHpWJdxVSrnfmw8mYEScqUP70cb0Q8X41uysJ1si6Eh1jYzXp9IE2DzOYsftYRyoCY9dJ/8QICgIcEun8D9PmAaBPlfT7lq4MFIlh61tYPiCswIHX+yBaOqT1QbuW7qpVQSv9lu6+xnvRVSlyopAypbGBTUdSalrSTaUBFYpInwUpxOzhti5TOdndyKhCGrdwAfBUcXIJB69p+Vw1egB76+n9q/h6ADglbf4LvnIHfF/981ODThF4m8HiS0riJVjQ6c+/EOZCYQfJrGrhBmPVNMmNArLKhQlkXWYqhbaxXY8ZNHphLuBJsZUEckCTFVHMgNKGJytIDeSUmw4QN4Qx9pReTgb3vYX/TCBuApf75f+P5Y4CRDdN+B+tngk8c8nt03CKGqipgd13OhotwOC5x9MCAknFFcmlmtPmagFFFYOCo0qRzXMhVi57pryNmIEqJlRi8bm52PfuNM8k4dfQv+4cO12l6zCGdg3jl730uE/KAPvS+f0wEAoAsA89/XfXQgBESIn6S5luDtiC8eh/YmIfpLqt1OMp5jXg8/24MveqUNUnPZsqw0Z3yVDldnaUOqIZfXlKrm36zzWhjRhaT+r+ncHI5/otUzfd2uSt7hl/bqXtoHaCC6+mqfrAOeoDD+PJ/xf8RgLMHfH/b8GeBihZIfSXidoQSJWB52NM1iRkzz3MkxpKPbUCrbDu5d5fgTAxkSK3JoEhYD1p2omere2LZTuqYLbdWa49Cx5Dww7tyXDUnioXRkHhwJyKFvd/AfPoYy4Fl7j1/LQorgEr9/X89+0qAOAwAf13sJoL8Gkd8wt25hWIp3Heez/eKODfPcSPCzpFNRDVqf7UlmnNQKGHgqd+jgVvJVm2f265QZTpLS5byur1tpT6ajvrHq3Q2MXWIxtUCehoj8YMk5LB9hRQegeTypn+nBQWA0QHgf7f2q4C5EFt+5ucOg2YfHXtq2SSHpS0ydnTL4IxFO6pvNb4ulBdInWfcsfSc7VMmXpSmE6eeXmZThJxpsgRohEfOk86+AHCoOpOMFsx1dv8s6oYT2k17uR7ngpXod34IEJqAaPfnfyABCIBZBpl/NPI2gTQVjX134x2ExSPMeR7VtYjZMWJ0W8ftjkA/YW1durCWykvjZFKu4p9LVwVbZKNkqpxh6U+6mRC2mGq2Q3SRvsIgcpc2sIpD0Bp4uiiFhW3ecXxOGgaCDe0Vf4cLPoDv+/5/mfw1gN4KKX+17emBqBmYfBHfVYUZKFR44NBtiv41bHJUwx+RJkP1apu2VJlkTwli4qrwoo1ax1dToNCtemRSTBGXz7kJbdM/PY/Dxht0dTLziH7Ul3loJEiE0uJsfdsVTYGL8Yt/AgcMgHYA7X8S+IqAYA+QfjzpxIIVHnp7tdqzhmAstXaxzEqMETpScGC/dJP3Rmdo8LIZnOVSEF+Opxumsl1sVF+dVrE5Z6NIiZSkvVdv2zsqjdnK8HVDLlyHyNjuegogM4NA5z9+YRG9gA722H97AgOA/gSyf43zCIHdE899yuTIg3ciNXpm1jmImTDwdJPITI4RPhRugbvslbFKt2Vfr/6eTFb4W1WkY6m6YPdQjJr2tNZp3EQlko7BgXHRNz2LAc+gdwMq7IUf3R58ohtFgrbr6n7hDFWAlPr8f/T9I4CECU9/De+vgVQY5nxh4POEzybJeCTS5YnCNAZzhsRzkP1Bsmu4t4aYU07nYuerA6KWWcJYO6HHrKJjaE3Zl624UWz/QOOPjcWHc7QzdIk40yl5tCWjhIDhJX0xF4CBMvBsf10IF4Ac//Z/bPlsgAcOwn6S6n6CwxzUewLcRoYaKzV38M23i9o493CNwL6S1UUuaQe0QpvbUfdfiqglpcRccFU+nkWwambASUiVfLyqbg49xY2eyWh1hy/Sh37XjHpaIYKD7OUEfrgS5IC09MV/1gMBgKMDyH/n9N6AhhINfh7mdoMoIZt6r9fAh1cvfHXNya6N4DzDbqi8K5WWSYlmbbAdnkpV6FxJpWSo1V8DUmGb3rMRaQBG2JJgwN9wCDnNi8HNI3dKK1aG0dvHe/UciIJf6rt+Og5wgDn59X9P/xWAKQhxf2XweYH+FjB9suGVhIMlOnlo02GJhTOdc7vFyo/TQGxs2Li7lz9NwmPurBihnVi7WSWiwKvGYntOpJiOt5drKUKMkFnE8HLxNPmJ9NG4eP8mAYUv4Np8hhi3gdruSX+3CSWAwP38f8f6UoCuDPF+6Os8gnAbKnxQ3d2F0imydzDPKIuiN5lxu8EKkrFE82kftW2az1DbYImpMqTUW3FWIJ83r5hl2koJlla7+m0+PmSOZcjcdMgwS4g11iZ6qCLUg5jkxn0QFA6BWvOvfzEFBIBHAtp/Qfa3gC4RSH5y5yeD2B/8evnYS4cULgR2CMsUja47cG/QvW6UeEhXZ3+xP51GVNVdP6Zpp+1eDFM5nMeySWghR4+TNL85cD46YIyCzKJ2kCzEhoTabXtGHs+CCemJfpMPjoDe9+t/qQALgM8Gj3++8UaBqRV2fQTjO4Q3JKd5r9TgiEYyMHTxxiWPpz8jbfq585YpTJpk960xoKFXsVoTo7yq6GGMTw==\" type=\"audio/wav\" />\n",
       "                    Your browser does not support the audio element.\n",
       "                </audio>\n",
       "              "
      ],
      "text/plain": [
       "<IPython.lib.display.Audio object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#hide\n",
    "from tsai.imports import create_scripts\n",
    "from tsai.export import get_nb_name\n",
    "nb_name = get_nb_name()\n",
    "create_scripts(nb_name);"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
